Forum: Author Hangout

Don't do what I did

Switch Blayde 🚫

I write novels as one file with chapter headings as H1 and have a ToC on the left (in Word) to easily jump to the beginning of any chapter. I also have a separate file with a list of characters and info about them, notes for the story, either from research findings or what I plan to put in the novel in the future, and a summary of each chapter written. Along with the chapter summary, is the number of words in the chapter.

When I edit a chapter, I update the number of words in the other file. I do that by using the ToC in the main doc to go to the chapter heading and highlight the entire chapter by scrolling down to the next chapter (holding down the left mouse button). Then I check the word count Word displays in the bottom left and go to the other file to update the word count for that chapter.

DON'T ever leave the document with anything highlighted. I did. The entire chapter. I don't know what I did at that point, but somehow I ended up doing something, hitting delete or some other key when switching back to the novel doc to save it, and Word replaced the highlighted text — the entire chapter. So I ended up deleting the entire chapter.

Now I'm an ex-DBA so I back up like crazy. The problem is, I just updated both files so I backed both files up not knowing the chapter was deleted. So the backup didn't have the chapter either. (I guess I could back up without replacing, but I would have too many backups and would spend an enormous amount of time managing my backups since I back up so often.)

So don't do what I did. Don't ever leave text in a document highlighted. The next key you press will replace it. A word of warning.

Replies: awnlee jawking Vincent Berg Michael Loucks Gauthier Uther_Pendragon BlacKnight

awnlee jawking 🚫

@Switch Blayde

Ouch! You have my sympathy.

AJ

Vincent Berg 🚫

@Switch Blayde

I'll tell you, the Mac's Time Machine feature has saved may ass multiple times, as it backs up EVERYTHING multiple times throughout the day (so your system NEVER slows down to a crawl), and you can pick and choose between multiple dates/times.

Unfortunately, as handy as it is, with my laptop, the machine needs to be running to back stuff up throughout the day, and then I need to plug in an external drive (which keeps disconnecting), so I still lose a LOT of files on that one. Plus, given the number of times I've updated the wrong files, or I've lost a HD (knock on wood, as I've yet to lose one on my Mac), and I can feel your pain.

madnige 🚫

As a DBA, surely you know about Version Control?

Replies: Switch Blayde

Switch Blayde 🚫

@madnige

As a DBA, surely you know about Version Control?

Yeah, but it's too much work.

Replies: Keet Vincent Berg

Keet 🚫
Updated:

@Switch Blayde

Yeah, but it's too much work.

Which bit you in the ass now. Decide how much you can live with too loose, say a day. Make daily backups. You don't have to keep them for years, just the last week and then keep a monthly backup for a little longer. I use a simple compress-and-copy system with different time settings for different sources and have not lost anything more than I allowed in over 20 years.

ETA: versioning would be the better option but make sure you have the versioned copies on a different drive AND backup those to an off-site location. Not the most well known but Mercurial is easier and works on Windows, MacOSX, and Linux.

Replies: Switch Blayde

Switch Blayde 🚫

@Keet

Make daily backups.

If I did that, versioning would make sense. But my hard drive crash paranoia causes me to back up many many times a day.

The other nice thing about versioning has nothing to do with losing data. It may simply be that you get to a point and say, "Hey, I like the way it was before better."

I will readdress my backing up. BUT I will never leave anything highlighted ever again. That's the lesson I learned.

Vincent Berg 🚫

@Switch Blayde

As a DBA, surely you know about Version Control?

Yeah, but it's too much work.

I agree, while it sounds logical, considering the overhead with minute, semi-insignificant changes (those which don't change the basic plot of the story (i.e. non-revisions), it's not worth the effort—until after you get into trouble.

Michael Loucks 🚫

@Switch Blayde

Now I'm an ex-DBA so I back up like crazy. The problem is, I just updated both files so I backed both files up not knowing the chapter was deleted. So the backup didn't have the chapter either. (I guess I could back up without replacing, but I would have too many backups and would spend an enormous amount of time managing my backups since I back up so often.)

Versioning backups (or local versioning) is your answer. I use Time Machine, another local backup, and a remote backup (all automated) and all three of them version. That's saved my ass on several occasions.

Automated services such as Backblaze offer version history and are basically 'fire and forget'.

Replies: Switch Blayde

Switch Blayde 🚫

@Michael Loucks

Versioning backups (or local versioning) is your answer.

I have a thumb drive in my laptop plus an external hard drive.

The external one is powered off except when I do my backups. This way nothing malicious can get at it.

Every time I update one of my spreadsheets I back it up to the thumb drive. Every time. I have that much fear of a hard drive crash on my laptop. Then I do my real backups to the external hard drive when necessary.

So every time I update my story in progress, I close Word and save it to the thumb drive. I do that many times a day; Some days, when I'm really into writing, it could be 20 or 30 times (or more) so versioning would be a nightmare. The problem is, I don't back up my work-in-progress to the external hard drive (which would be a second version).

But my point wasn't about backups. It was about having something in your word processor highlighted and leaving it like that. In my case it was a whole chapter. But whatever it is, all it takes is a keystroke, like the space bar, and, poof, what was highlighted is replaced with a space.

Replies: Michael Loucks Vincent Berg

Michael Loucks 🚫

@Switch Blayde

But my point wasn't about backups. It was about having something in your word processor highlighted and leaving it like that. In my case it was a whole chapter. But whatever it is, all it takes is a keystroke, like the space bar, and, poof, what was highlighted is replaced with a space.

And yet versioning backups would have saved you, especially if they are automated. I've done what you've done and simply gone back a few hours in my backups. My writing is done in BBEdit, a very extensive plain-text editor mainly meant for programmers, but perfectly suited to writing. My habit, learned at work ages ago when MS Word and Windows had nasty habits of crashing, is to save after every paragraph (sometimes after every sentence). Those files are saved to 'Sync' (a service like DropBox, except fully encrypted so the server operator can't see my files). That automatically and immediately backs up my file as soon as it's changed, and I can go backwards in time. Once an hour, my automated Time Machine backup runs and makes a copy on a network disk in my house (though it could be a local disk, too). Also once an hour, my automatized Arq backup is made to a cloud server (fully encrypted).

All of those things happen automatically and setting them up is fairly straight forward. I could make my cloud backup instantaneous as well, but because I use 'Sync' I don't need to do that.

In my setup I have a minimum of eight copies of any file, and at least six versioned backups, and at least three offsite. But I'm paranoid. :-)

Replies: Keet

Keet 🚫

@Michael Loucks

In my setup I have a minimum of eight copies of any file, and at least six versioned backups, and at least three offsite. But I'm paranoid. :-)

Not paranoid, you're doing it right. Maybe a little too much for most authors here but doing half of it would save most of them from loosing too much of their work.

Vincent Berg 🚫

@Switch Blayde

Every time I update one of my spreadsheets I back it up to the thumb drive. Every time. I have that much fear of a hard drive crash on my laptop. Then I do my real backups to the external hard drive when necessary.

I'm not quite that obsessive, as I make a LOT of changes to a LOT of files during the day. While Time Machine backs up multiple times during the day (though I've still lost changes). I frequently back up to one thumb drive, but not as soon as I make changes, as I'll typically only do it once or twice a day. As for storing to cloud sharing sites (DB), I typically use that for editing (files to be edited and my editors suggestions), otherwise I only save completed books or initial drafts there.

But, speaking of cloud storage, version for every one of my stories, for every minor change to a file, would entail monthly subscriptions, which I try to avoid. DB is free, and while I wouldn't mind increasing how much I can save, ALL the Pro levels include limits on what I'm allowed to save, so I've never bothered.

Replies: Keet

Keet 🚫

@Vincent Berg

...ALL the Pro levels include limits on what I'm allowed to save, so I've never bothered.

How do they know what you save? I would assume that you first encrypt your work before storing it onto someone else's computer. I suspect you mean how much you can store. I bought your box set from Bookapy so I know you use extensive graphics that would greatly increase the required storage space. Little you can do about that.
There's a little dirty trick you can use. Most cloud storage providers give extensive discounts for the first year. Since there are plenty of options switch every year with a new discount.
By-the-way, a thumb drive is not the most reliable backup storage. They tend to fail fast if you use them extensively.

shaddoth1 🚫

:( i have that T-shrt. definitely not a fun experienec

Shad

Gauthier 🚫

@Switch Blayde

Good advise.

I also see a lot of early typist (those looking at the keyboard) typing ctrl+a instead of shift+a with the same result.

In the same vein, lock the session when away, you don't want to know how effective at file deletion a baby or a cat can be.

If you are using windows, try the free ShadowExplorer
The shadow copies (aka restore points) made by backup programs, installer and Windows Update also contains users data files.

Next time :
Turn on file history.
Increase disk space allocated to system protection. It default to 2% on fresh install.

Replies: Switch Blayde

Switch Blayde 🚫

@Gauthier

typing ctrl+a instead of shift+a with the same result.

Yeah, I can see that happening.

I'm using Word on a Mac and it does something I don't remember happening on the PC that results in deleted data. I don't know exactly how it happens, but when I highlight something to change and change it or delete it, Word deletes the word before it. I have to remember to have a space between what I'm highlighting and the word before it. I think that's the scenario. But somehow I keep losing words.

Replies: Gauthier

Gauthier 🚫

@Switch Blayde

But somehow I keep losing words.

https://superuser.com/questions/962710/how-to-make-microsoft-word-selection-behave-like-it-would-in-a-plain-text-editor

Replies: Switch Blayde

Switch Blayde 🚫
Updated:

@Gauthier

https://superuser.com/questions/962710/how-to-make-microsoft-word-selection-behave-like-it-would-in-a-plain-text-editor

My Word is different. There is no "Options" under "File".

There is, under Word, a "Preferences" and then an "Edit." Under that I can uncheck the "select whole word" but there is no "smart paragraph" to uncheck.

ETA: I just read more of the article and came to the Mac part.

ETA2: That fixed it. Thanks.

Dinsdale 🚫

I'm currently looking at https://www.duplicati.com/ as backup software.
There are two versions available: 1.3.4 is old and unsupported, 2.0 is marked as being beta. All the indications I have seen point to the beta version being rock solid.
The Windows version requires .NET 4.5 (no big deal) but the Linux and OSX versions require "Mono" which is a port of .NET to Unix.
Initial Backups backup whatever you tell them to, after that they are incremental - even down to only copying the changed parts of large files. All backups are compressed and can be encrypted.
https://duplicati.readthedocs.io/en/latest/01-introduction/ has quite a bit more, although I'd be very hesitant about using Amazon Cloud Drive because Amazon are allegedly discontinuing access to their API.

damoose 🚫
Updated:

While the thread started as an... opps... I left some text highlighted and had a fumble finger that was then compounded by the bad copy of the file being saved over the top of the good 'backup' of the file. In addition, I just read that aroslav lost a bunch of work due to a liquid incident with his writing impliment of choice. I have been following this thread with great interest.

While I am not a writer of erotic fiction, some have said my process documents are works of fiction, horror to be exact. However, I would still like to offer some suggestions if I may. The focus here is on 'home' usage, not enterprise and single machine, not the whole home. Given this focus, there seems to me that there are some clear requirements here concerning backing up copies of a creator's work.

First, it cannot be too difficult, either in setup or usage. Second, it must inexpensive as not all have the same means. Third, it must be cross-platform and writing tool agnostic. Fourth, it must be automated. Fifth, encryption support. Finally, it must be bandwidth friendly.

One other thing, best practice says you should have 3 back up copies of the data, one local, one remote and one off-site. I'll not tackle the best practices for how many 'versions' of said back ups should be kept, that is a whole topic in and of itself that is not needed in this context.

OK... so here we go...

I have seen both Arq and Duplicati mention here. Both are excellent choices for this scenario. They are cross-platform, they both run automatically on a schedule, they both support deduplication, they both support, local, remote, and off-site backups, they both encrypt. I suggest you choose one or the other.

A very, very simple explanation of dedup is that it is a technology by which any single give 'chunk' of data is saved only once even if it used in multiple versions of a backup. This means that the tools will only 'back up' new chunks which makes it very bandwidth and space usage efficient. The big bandwidth hit is the initial back up where everything must be copied to the destination the first time.

Arq is paid software and Dupicati is open source. Arq costs money, 50$ US, one time fee, but that also provides a known level of support. For an extra 7$ US a month you can get 1TB of cloud storage from Arq. Duplicati is free but support is community based. I use a cloud service for off-site storage, not Arq. I use both Arq V5 and Duplicati 2.0.4-Canary and both work as expected.

For ease of use, I would choose Duplicati actually, but both provide a rich UI with which to interact with the tool. Arq is its own application while Duplicati is browser based.

Both allow you to choose what you want to back up and both provide the ability to create multiple data sets. So you can choose to back up all of your documents folder or some subset of it. You can choose to back up documents and photos folders, or any combination of folders and sub-folders really.

Both have a built-in scheduler that will allow you to specify a recurring time or specific time or both to execute a back up run. Both allow you to choose how many copies you keep, you can limit the bandwidth you use, you can use all the major cloud environments or not.

The basic scenario I follow is, a single backup set, backup every hour and keep at least one copy of any given file. I do not do a local backup, only a remote and off-site. I use a NAS (remote) and Backblaze (off-site). I use a NAS because it sits on my home network, but for you laptop road warriors, you can use a USB drive, either spinning rust or SSD in place of the NAS, but not a USB key, dongle or stick, they are very, very prone to failure when used continuously. I have email alerts set to send on error only.

You can use any supported cloud vendor for your off-site Arq and Duplicati provide a list of supported vendors. I use Backblaze B2, think 'cold storage', rather than hot storage like OneDrive or Google Drive. Why? Because this is back up not an extension of my local drive storage to the cloud, so I don't need the 'speed' of a OneDrive for back ups. Backblaze B2 storage costs me $1.70 US per month for 250 GB of storage. I was up to $5.50 US for a TB at one point in 2019. B2 is usage based so you pay for what you use rather than a chunk of space you may or may not use. I am sure you can find other cloud vendors with similar storage, Amazon Glacier comes to mind.

As a generalized guide, use a cross-platform tool like Arq or Duplicati with the ability to dedup and have multiple 'copies' of the back up. Use a back up tool that is automated so you never have to 'remember' to compress and copy. Use a USB drive or other attached storage for 'remote' backups and cold storage in the cloud for off-site. Don't get me wrong, you can use platform specific tooling, like Apple's Time Machine to great effect, but platform specific is platform specific. ;-)

Finally, back ups are only half the battle, you must also restore from your back up. It does you no good to back up and then not be able to retrieve that data. As a general rule, I would run a verify at least every 3 months or so if there have been no errors reported during back ups and a restore of something every 6 months. As always do as I say not as I do. :-)

Apologies if I have overstepped with this note. And I hope someone will find it helpful.

Later

Damoose

Replies: Dinsdale

Dinsdale 🚫

@damoose

Aroslav may not have lost his data. His computer (laptop) is pretty much toast but the contents of the hard drive may be salvageable.
Having said that, Duplicati would have helped him keep his USB backups current.

Tw0Cr0ws 🚫

In Linux there are file systems that allow the user to roll back to an earlier snapshot of the files.
Kind of a shame that the OSes that cost money are so much less advanced.

Replies: Michael Loucks

Michael Loucks 🚫

@Tw0Cr0ws

In Linux there are file systems that allow the user to roll back to an earlier snapshot of the files.
Kind of a shame that the OSes that cost money are so much less advanced.

APFS on Macs has snapshots (and I've used them):

https://www.lifewire.com/roll-back-apfs-snapshots-4154969
https://blog.macsales.com/56681-working-with-macos-snapshots/

Windows supports snapshots as well (though I haven't used them):

https://cloud.google.com/compute/docs/instances/windows/creating-windows-persistent-disk-snapshot

Replies: damoose

damoose 🚫
Updated:

@Michael Loucks

I guess this should be replied @Tw0Cr0ws, but I replied @Micheal's post... sorry 'bout that.

I have used snapshots on Windows. They work, kinda. They are short term copies of artifacts which can help you if you realize that something is wrong within the snapshot time/space window. Snapshot is 'free' in that the functionality comes with the O/S. But snapshots are not back ups, no matter the O/S.

Creators can and should take advantage of everything that can safe guard their intellectual property and in some cases their livelihood.

Later

Damoose

Replies: Michael Loucks

Michael Loucks 🚫

@damoose

But snapshots are not back ups, no matter the O/S.

I don't disagree - I was countering @TwoCrows comment about them only existing on Linux. They are awesome for rolling back failed updates, but they are not, as you say, backups.

Uther_Pendragon 🚫

@Switch Blayde

Can't you hit "undo"?

That has saved my ass several times.

Replies: Switch Blayde

Switch Blayde 🚫

@Uther_Pendragon

Can't you hit "undo"?

I wish it had been that simple.

Not realizing what I had done, I had already saved and closed the file. You can't undo it after that. And before I realized what happened, I copied the bad file over my backup.

BlacKnight 🚫

@Switch Blayde

So don't do what I did. Don't ever leave text in a document highlighted. The next key you press will replace it. A word of warning.

The terminal-mode text editor I write in doesn't work like that, so. You have to use Ctrl commands to mass-destroy data. Simply typing without hitting the Ctrl key will never do it.

The largest data loss I've ever had was the time the laptop I was writing on shut down just as I hit save (I suspect the extra power drain from spinning the hard drive up to write to it was what killed the battery). When I got it back on mains power and booted up again, I discovered that it must have crashed just as it started to write the file, because the file existed on disc, but it was empty, zero-size.

I had backups, but I'd written something like 8k words since that morning's scheduled backup. Most of that I was able to recover from the browser cache on a different machine, but I still lost about 2k words that weren't stored anywhere.

The ironic part is that I'd been saving regularly, so if I hadn't tried to squeeze in one last save before the battery died, I would have lost maybe the 20 words of the last sentence I'd written.

Reply to topic

Forum: Author Hangout

Don't do what I did

WARNING! ADULT CONTENT...