Linux Passwords Cracked

Questions, answers, and news related to computer hardware and software

Re: Linux Passwords Cracked

Postby Kellemora » 16 Dec 2012, 14:01

I agree Yogi, for me it has always been complex. I have a hard time understanding these confounded computin' contraptions.

When I use the term backup, I use it incorrectly, because in my case, a backup is NOT a backup set, but an exact mirror of my data.
And in many cases, the mirrored drive IS one with an OS on it, that can be plugged into any computer, either using the OS on the drive, or as a slave so just the data can be read.

In my daily work, rather than waiting for the cronjob to run rsync at night, I always copy my changed data folders to the NAS AND to an External HD. Around 2AM cron runs rsync to copy data from the computer to another encrypted external drive. Then around 3AM that encrypted external drive is mirrored to an external drive in my house, and around 4AM it mirrors to a drive in St. Louis.
I have it set so that it only copies changed data.

Now you know, this type of method WILL copy and delete corrupted files, destroying all possible recovery. SO, I maintain monthly and weekly redundant files that are manually copied to separate files, without deleting previous files.

I'm looking into ZFS, but it seems I should wait until they have more bugs out of it first.
I have used it already, to make use of some old HD's I had laying around. But quickly learned that what you read about RaidZ is not exactly what you get. Although you CAN use any size HD in RaidZ, it drags your capacity DOWN to the smallest drive size. UNLESS, yes, always away around those problems. I forget the proper name for what it's called. But you build Separate Banks of Arrays. For EG: Put all those old 50 gig drives on a bank of its own. All the 100 gigs on a bank of their own, etc.
I did that for awhile testing out RaidZ, but my volatile data, although still on them, is not considered as a backup, just experimental.

My biggest problem in the past with having so many redundant backups, was I ended up not remembering this when I had a stack of HD's sitting on a shelf and the post-it notes all fell off. So rather than be safe than sorry, I copied all of them onto the larger drives, then tried finding duplicate files. Checking for duplicates DID NOT insure the files were duplicates and I ended up deleting numerous files that were not identical. They may have had the same name and the same checksum, but were in fact, totally different files. So some of my things were lost when I did that. Hopefully nothing to important.

I've also mixed up my web site files in the past a few times. Partly because of laziness and then forgetting where I saved my daily work. I may make several changes on my computer, with the intent of saving that folder to the External Drive, but it gets late and I copy it to the NAS. The next time I work on the file, I draw it from the file server, make a change, then save it to the external HD.
What I did previously was not saved where it belonged, so although I have them on the NAS, I will now have to go back and change the File Server Copy to match it, check it against what I saved on the External, and HOPE I didn't mess anything up in the process.
I should mention that our main file server died, and rather than building a new one, I began using the External HD mirrored from the File Server as our File Server using Sharing. Worked OK and I changed the cron job to backup (mirror) the external offsite.

I guess what I'm looking for is a way to insure I won't lose data, forget the name for that too, shadow data loss?
I would like my system to automatically keep redundant copies that are not overwritten, and that is why I'm looking into ZFS.

Just maybe, it might be wise for me to consider using a Hosting Service? Although I'm leery of things going out on-line.

Thanks for your input Yogi!

TTUL
Gary
User avatar
Kellemora
Brainiac Class Poster
Brainiac Class Poster
 
Posts: 3389
Joined: 07 Jul 2012, 18:52
Location: From St. Louis, current Knoxville, TN

Re: Linux Passwords Cracked

Postby Yogi » 17 Dec 2012, 10:48

Well Gary, I'd be lying to you if I told you that I understood even half of what you are doing down there. I know you have difficulty with the technical terms and that you and I might have different words for the same thing, but it goes beyond that. Preserving your data, as with maintaining security, can be done in nearly an infinite number of ways. I think you have the basics down, but perhaps you may benefit from streamlining your operations.

I'll give you an idea that is scalable and fairly simple in design. As a system administrator from way back I know the value of automating processes and avoiding manual operations whenever possible. I'd recommend that you automate your backups as much as possible so that you can concentrate on actual work instead of maintenance. Both Windows and Linux have software to automate processes, and you can do it without running scripts, i.e. crons. Then again if you are really into customizing and short on change, go crazy with the cron jobs.

Anyway, your day to day business should be backed up every 24 hours. Thus all your work station files can be backed up nightly while you slumber. Incremental backups are fine at this point because all you want to do is keep current. Thus your daily backups go to Server_1 which probably has some sort of RAID storage. I find it hard to believe you would need more than RAID 3, but if you have the resources, go nuts. You can never have too much storage nor be over protected.

Your daily backups from Server_1 should be copied completely (no incremental backup) onto another server, Server_2. Do that once a week. At this stage in your archive farm, RAID isn't really necessary for this server. Thus Server_2 gets used once a week to back up all the files from your daily storage on Server_1. Server_3 would be used once a month to back up the contents of Server_2, and only need be on line once a month. Server_4 would do the quarterly backup of Server_3. And, you can continue this chain on to any scale you deem appropriate.

You really don't need hardware servers to build your archive chain. Simple memory banks of hard disks (not SSD) will do just fine in place of a full boat server. However, if you want to create a mirror at any stage along the archive chain, that will be the point where you will need server software to generate it. I suppose the best place to create an off site mirror would be right at the daily backup stage. That way disaster recovery is simply a matter of switching over to the remote mirror site in an almost seamless operation.

The above takes care of your data files and assumes you will switch to a mirror site in the event of a disaster, or be able to reconfigure your primary work stations quickly enough to not interrupt business significantly. As you know, rebuilding a system from scratch is nothing you want to do in an emergency situation, thus in addition to the file archives you should have system images readily available. An image copies the entire contents of your work stations bit for bit. Thus recovery consists of copying the image to a clone of the original hardware, and voila. Since you have a server farm for your archived files, you really don't need your NAS for anything other than a place to store your work station software images.

The beauty of the server archives is that you don't need them to be on a router. Ad hoc point to point connections would be all you need to back up the backups. A router would be necessary, however, to negotiate any connections to a proxy server and/or NAS.
User avatar
Yogi
Oracle Class Poster
Oracle Class Poster
 
Posts: 7013
Joined: 04 Aug 2007, 19:37
Location: Chicagoland

Re: Linux Passwords Cracked

Postby Kellemora » 18 Dec 2012, 11:51

Thanks for your in-depth explanation Yogi, much appreciated.

I think I'm already doing what you are instructing me to do, I just didn't word it very well.

Ignore the NAS as far as being a part of my redundant backups.

Before my File Server crashed, all computers were sorta used as Dumb Terminals. With the exception that the computers had the working programs, and only DATA was drawn from and saved to the File Servers drives. Now that I no longer have different people working on the same files, using a File Server and multi-user programs are not really necessary.

When the File Server was in operation, if someone opened a file and did some work, when they saved their work it was back on the File Server.

Today, in lieu of a dedicated File Server, I have a single computer with a large external HD connected to it, Not Raid. Only the folders each person does their work from are Shared, not the entire EHD.
As an aside: This computer is connected to my KVM switch, so I can access all folders on it, including those not shared.

Now: In normal practice, when someone goes to the EHD via the LAN, and opens a folder, that folder appears on their desktop. Thus the changes made in that folder on their desktop do not appear on the EHD until they manually save the folder back to the EHD.
That is the part I do not like about this set up. They may forget to UNMOUNT the Folder.
I'm requested to go in and make some changes to a file. So I open the folder, make those changes, and save it back to the EHD and unmount the Folder.
They go back to work on the existing folder on their desktop, and then save it, thus deleting the work I did. Or they complain that I never fixed what they asked me to do. Either way, it is then a mess that has to be redone.

I'm not all that computer literate when it comes to things like that. Took me awhile to figure out, data was not being saved, unless we saved the folder back to the EHD when done working on it, AND unmounting it, so we had a fresh copy when we started to work on it again.

On the backups: The EHD everyone works from is in my detached Garage Office and accessed through the LAN via the computer the EHD is connected to, all they see from this EHD are the Shared Folders and nothing else.
The nice part about using RSYNC to copy the EHD in the Garage Office, to a second EHD located in the House, is that it can be set to Copy Only Files that Changed, since the last time RSYNC was run. If you deleted a file or line in a document, it WILL BE deleted from the mirror. In other words, it is not sequential. What is on the Garage EHD is Identical to what is on the House EHD. If a file on the garage EHD becomes corrupt, that corrupted file will overwrite the house EHD file with the corrupted file. One of the reasons I'm looking into ZFS.

To prevent sabotaged or corrupt files or accidentally deleted files from being permanently lost, I use a second very large EHD, and a Windows backup program that saves changed files sequentially, without deleting anything, until the drive becomes full, then the oldest backup folder or two are deleted to make room for the new. Only specific folders are backed up to this sequential file, namely the accounting folders, and other master volatile data folders. Non-important folders are not backed up sequentially to this drive.

The House EHD (encrypted mirror of garage EHD) is copied to an encrypted EHD in St. Louis. This is done from a Windows XP-Pro-MCE computer to a Windows XP-Pro computer, the commercial program that handles this is on the St. Louis side. So I don't actually send the files to St. Louis, it draws them from my house EHD via the computer on a set schedule.
I think it goes without saying that the EHD's do not contain Programs or OS's, only Data.

RSYNC does not have it's own Clock Timers. Nor can it self-change what you tell it to do.
So I set up RSYNC to backup the garage EHD to the house EHD, and use CRON to start it running. It signals CRON when it is finished and shuts down RSYNC.

Now, as far as the NAS goes. I do have another copy of RSYNC that copies the garage EHD to a Folder on the NAS, as a secondary backup. But we mainly use the NAS to save folders we are working on, until we reach a point where we or mainly I, decide to save it to the EHD that gets backed up. My wife downloads audio books and Kindle book to her computer, then from her computer to the device she listens to or reads them on. These types of files are never placed on the File Server or EHD, as they are temporary. However, books without DRM's or books she buys, these are stored in HER folders on the NAS. I also have a backup program on her computer that copies certain folders to a specific folder on the garage EHD, so they end up being backed up to all EHD's. But as redundancy, all of her computer is backed up to a small EHD connected to her computer, not redundant elsewhere.

I never had luck trying to restore a computer from a crash using backup sets. Eons ago, I used Tape Drives to back up each computer in the office. When one died and we replaced it, even though I restored it using the Tape Drive, they never would boot up. So I always had to install the OS, then each program, and finally could restore from the Tape Drive to get all the files back. It was very time consuming. For this reason, I quit backing up the OS or Programs and only backed up the Data Files.

Eventually I learned about ISO's. So after I had a computer put together with its programs, I would attempt to burn an ISO only to find, they had grown to big for even a DVD. So I began saving them as a file on an internal hard drive, then copying that to an external hard drive I kept on the shelf. I could then delete it from the smaller internal hard drive until such time as I needed to place it back on a new computer. Now although it would install and everything be back to like new, the upgrades and updates may no longer be available, or the wrong drivers for the new computer, etc. So I still ended up with a mess to clean up.

So far, knock on simulated wood grain, by using VB or VM and making a copy of the Folder, it is a simple matter to put that folder back and everything Just Works Right. Provided I make a new copy after each upgrade or update. Which can sometimes mean doing it at least once a week.

I have to laugh. I have accounting program data saved, that is now so obsolete, nothing can read it anymore.
That's one of the HOOKS I hate about Proprietary Software! Like Quickbooks Pro!
Unless you take ALL of your Archived Data Files and Convert them to a new version. After a few versions of upgrades, they no longer support reading the older files. I had this problem big time with Family Tree Maker too. Fortunately, I kept the old original install programs, so could access the data. However, I had to print it all out on paper and manually type it all back into their newer program. Trying to convert to GEDCOM lost over 90% of the informational data. And new programs didn't align the Field Tags properly. Place of Burial, may come out on the new program to be Place of Birth, hi hi........ Makes for a funny read, John Doe was Interred in plot L7X456 at graduate of St. Mary's Hospital on 8 pounds 3 ounces at 792 Humphries Boulevard, class of 1965.

TTUL
Gary
User avatar
Kellemora
Brainiac Class Poster
Brainiac Class Poster
 
Posts: 3389
Joined: 07 Jul 2012, 18:52
Location: From St. Louis, current Knoxville, TN

Re: Linux Passwords Cracked

Postby Yogi » 19 Dec 2012, 10:20

It appears that you need a good system of "version control" in your business. :grin:

I'm not sure there is software to accomplish that task, but there probably is. Basically you need one master copy of each document. The master is made read-only and can be downloaded by any of the collaborators. The master should have a header with a list of all changes made, by whom, and when. The collaborators would add their comments to the header when they are finished editing. Only one person should have read-write authority for that master. All the updates made by collaborators could be put into a directory or perhaps even a server for that purpose. Thus there will be no overwriting the current master. The editor of the master copy would have the responsibility to be certain that all the updates were applied at the end of the day.

To reiterate, in the above scheme the document being edited would be downloaded to a shared "editing" directory instead of to a workstation desktop. A serial number could be added to the title of the downloaded version just to keep things straight when more than one person is working on the same document. I'd download it as Document-001.ods, and if you have a need to edit it you could not save it under the same name. Therefore, you would download it as Document-002.ods. The editing directory could have several versions of the same document all independent of each other and not threatening to overwrite the master. Nobody has to worry about esoteric things like mounting and unmounting. It's all taken care of by transparent system functions.

At the end of the day the master copy is backed up to the proper server. After the masters' backup is completed the editing directory is emptied and edits are applied to the new master. This solves the problem of overwriting without backing up the document first, but it does add to the workload of the person maintaining the masters.

________________________
As far as your old business data is concerned, it might be too late to standardize it. I'm sure it can all be put into a commercial database such as Oracle or even MySQL. If Quicken or the others can't work with an independent database, then it's time to find some new software. The beauty of Oracle type databases is that it's portable. The downside is the cost of the license. MySQL might be a more cost effective approach in that case. If you can't convert to a common format, then you may just be doomed to supporting old software for old data. Just because there is something new and better out there does not mean you have to use it.

________________________
Disaster recovery should not be a problem. Keeping copies of your working desktops in virtual machines is a beautiful solution. When you save a .vdi file, essentially you are saving a complete operating system. This is also the case when making system images in Windows OS's. It's just a matter of putting the .vdi or image in place to recover from disaster. Simple backups of files, even critical ones, will not yield a bootable operating system. I have a feeling that is why you have never been able to do it. You simply have not copied everything needed to make the recovery.
User avatar
Yogi
Oracle Class Poster
Oracle Class Poster
 
Posts: 7013
Joined: 04 Aug 2007, 19:37
Location: Chicagoland

Re: Linux Passwords Cracked

Postby Kellemora » 19 Dec 2012, 20:42

Hi Yogi
What you described in the first couple of paragraphs, sounds almost identical to how the File Server used to work, before it died.
It maintained Originals as Originals and/or first drafts, and all changes to it were synchronized as a separate file. Or extra files if the changes overlapped the same areas and then required manual adjustment before additional editing could continue.
If there were no problems with the synchronization, the second draft was rendered read only and any further changes became yet another new file.

On the next topic, I've tried MySql, it's over my head. Not needing to use QuickbooksPro anymore, I'm now using GnuCash, which is more powerful than Quicken, but no where near close to Quickbooks. I don't like it as well as I did Quickbooks, but the data is transportable and printable in plain text.

On the third topic, I used ISO's so they would be bootable. But now since I use Linux almost exclusively, just copying the /home directory is often sufficient to save everything and restore everything, regardless of the Distro used, if compatible that is.

On really ancient accounting data, some as far back as 1977, I don't think I really need to get back that far anymore. I have over the years kept archived information in plain text or generic data files that can be read by anything. But some things I was not able to save that way. Such as the first and ongoing Owner Contribution to the original start-up company.
Because of ancient original documents considered important, an accountant I used for a few years suggested closing out the sub-companies, creating a new parent company, then reopen the active sub-companies. This established a new starting point for everything concerned. We closed out the original start-up account, I cashed out, then placed it all back in the new parent company. That was back in 1984 or 1986, and all of my records since that time are impeccable, currently readable, plus each quarterly and annual report and an annual hard copy printout of all the books is on file, scanned and filed as a locked PDF. Backed up redundant of course.

In my old age, I have dumb-ed down almost everything. Other than my accounting for my AZ-NO3 business, the only other accounting has to do with my writing and publishing books. All of the rest are just document and image files, either for my writing or for personal reasons. Like our family photo's, genealogy and the like.

Another friend has just about convinced me that using the Cloud is less likely to be hacked than my own computers. But he's not really pushing in that direction as much as using the slow NAS I have, in place of my old dead server, and backing it up to a simple external drive, which would still be mirrored off-site. The backup would not be Raid, and a copy of the files, not a backup set, so it would look just like the HD on your own computer. Not much different than I'm doing now, only eliminating one of the redundancy steps. Or the way he put it, since I am copying my data to an off-site location anyhow, which is good, I'm just wasting my time and resources copying data from one building to the next building at the same location.
And here I thought it was a good idea! If the house burns down, the data is safe in the garage, or vice versa.

FWIW: To copy ALL of the data to a new external drive, when both drives are connected to the same computer, the time it takes is usually less than 5 minutes. To copy ALL the data over the LAN to a new HD on a computer in another building can easily take 3 to 5 hours. Now if I try doing it to the slow as molasses NAS it takes over 5 hours. The NAS apparently has a top rate of 100 mbs, not 1 gig as the rest of the LAN. To send ALL the data out-state over the internet, at one time, would probably take well over 10 hours if not full 24 hours. When I first started storing out-state, I didn't have that much data, and have only added to it over time. I've never done a complete new fresh transfer. When I view the out-state HD, all the files are identical to what is on my local HD's, so I've never seen a reason to start over new. Now we have replaced the HD back home with a new much larger one, but they took care of copying the files from one to another, and I think they may also keep a redundant backup of that drive. I'm not really sure what they have there, it is a fairly large company, doing this as a free favor for me. Just like I have not paid for my four St. Louis e-mail accounts now in over 10 years. They say it is no cost to them to keep them up and available to me.

OK, I know your time is valuable. I appreciate all of your comments and input. I'm going to study everything you've said again, plus what options I have to simplify my mess and extra labor steps.
Thank You very much for your wealth of information.

TTUL
Gary
User avatar
Kellemora
Brainiac Class Poster
Brainiac Class Poster
 
Posts: 3389
Joined: 07 Jul 2012, 18:52
Location: From St. Louis, current Knoxville, TN

Re: Linux Passwords Cracked

Postby Yogi » 20 Dec 2012, 08:00

All I can add here is that I've done all the above in a Windows environment, except for creating mirrored servers. I"m sure Linux has similar, if not better, software to accomplish the backups and lock down the security of your network. I need to keep things organized, which means I need to understand what I'm doing so that I can derive solutions to problems once I forget about how I set it all up in the first place. The underlying principle in just about all the technical projects I've been involved with is simplicity. Keeping it simple has saved me a lot of headaches. Then again, you might argue that I have a simple mind. :lol:

I'm sure your business information system can be streamlined, but you got to ask yourself if it's worth the cost and effort. In the business world it all boils down to the bottom line. From a perspective of keeping your mind fresh and learning new techniques, experimentation and curiosity are your friends.
User avatar
Yogi
Oracle Class Poster
Oracle Class Poster
 
Posts: 7013
Joined: 04 Aug 2007, 19:37
Location: Chicagoland

Re: Linux Passwords Cracked

Postby Kellemora » 20 Dec 2012, 10:35

Since my heart attack, most of the things that used to be important, no longer are.
I still have sitting in my warehouse, all the components necessary to build a new, more advanced bottling system.
My old one is breaking down, one filler head at a time. Becoming more of a PITA to use with each order.
The reason it sits, has more to do with a design change they made to the filler heads, they replaced the nice conical tip with a fairly straight tip. So I will have to make considerable modifications to them before they will work properly.
The stir units, which look like twin squirrel cages, came without mounting frames, and needless to say, my existing mounting frames are of a different shape.
It will probably take having to fill the bottles by hand, before I get around to working out the problems with the new equipment to make it functional.
The same basically applies to my computers. I use them while they work, make sure my backups are running, and hate having to change everything, even when it simplifies the whole process.
For security reasons, I power down the bank of external hard drives when I stop for the night.
Many days, I don't bother to power them back up and just save my work to the NAS.
So in the mornings I'm faced with error messages telling me the backup failed, no source files or drives found, etc.
I kill the messages, go do my work, and if I do things really important, I will fire everything up, save my work from the past few days, and then manually start the backup services.
Since I went back to work full-time, I have almost no time to do the things I really liked to do. Or do the things I should be doing.
Plus add to that, because of my heart attack, a lot of the things I liked to do, I can't because doing it would kill me.
This means I usually have to pay someone to do the things I used to do myself, with ease, and it costs a lot more than it should, besides never being done the way I would have wanted it done.
In short, I went from having time to piddle away playing around, to no time to even get the necessary stuff done.

TTUL
Gary
User avatar
Kellemora
Brainiac Class Poster
Brainiac Class Poster
 
Posts: 3389
Joined: 07 Jul 2012, 18:52
Location: From St. Louis, current Knoxville, TN

Previous

Return to Personal Computers

Who is online

Users browsing this forum: No registered users and 1 guest

cron