- Unit testing (Re: Putting project code in many different files) - 22 Updates
- Commenting code considered harmful - 1 Update
- cmsg cancel <n8u511$mog$2@dont-email.me> - 1 Update
- You have to believe in our beloved Jesus Christ. - 1 Update
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 07:42PM -0500 On 2/3/2016 4:49 PM, David Brown wrote: >> with other users. Your disk won't even provide that speed. > Jerry, have you ever heard the phrase "when you are in a hole, stop > digging" ? You are truly making a fool of yourself. Yup, and you should have stopped several posts ago. > re-copying all the data? You copy the /changes/. Unless you are > producing a movie or running a particle accelerator, you don't make > terabytes of /new/ data every day. Backing up 100MB every 15 minutes comes out to about 9.6Tb per day. Copying the changes means you have to restore starting with the last full backup. That may have been weeks ago. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 07:43PM -0500 On 2/3/2016 6:32 PM, Ian Collins wrote: >> fiber, you won't get 100MB/s throughput on a lan - especially one shared >> with other users. Your disk won't even provide that speed. > Weren't the capitals *WAN* bold enough for you? Yep. And I stand by my statement. > Besides, all my systems (home and on client site) use 10GE (something > you may not have encountered in your 80s world) and the storage can > happily manage 4-500MB/s machine to machine. For hundreds of megabytes? I doubt it. What disks are you using that are that fast? >> Not to mention needing several terabytes per day for all of the >> replications. > Google "incremental". Google "restore from incremental backup" -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 07:46PM -0500 On 2/3/2016 5:40 PM, David Brown wrote: > directories, or to roll back the virtual machines, or to make temporary > duplicates of the entire virtual machines. Pick the right technology, > and it's not hard or time-consuming. You haven't? I have. In fact, I've been using incremental backups since the mid 1980's with DB2. A database, but the concept is identical. If you've never had to restore from snapshots, you have absolutely no idea what you're talking about. To restore from snapshots, you need to go back to the last full backup, then restore each snapshot - *in order*. One snapshot missing or corrupted? EVERYTHING after that is lost. And if you try to restore later snapshots, you will have a corrupted repo - one where you have no idea what is good and what is bad. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 04 01:49PM +1300 Jerry Stuckle wrote: > Backing up 100MB every 15 minutes comes out to about 9.6Tb per day. > Copying the changes means you have to restore starting with the last > full backup. That may have been weeks ago. Bong! Wrong again. I restore from the appropriate snapshot. Each snapshot represents the state of the filesystem when it was taken. If you are replicating from a busy build server (or a VM image) you may get many megabytes of data every 15 minutes, but nearly all of that will be changes to the same files, the the aggregate isn't much bigger the the individual replications. -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 04 01:59PM +1300 Jerry Stuckle wrote: >>> with other users. Your disk won't even provide that speed. >> Weren't the capitals *WAN* bold enough for you? > Yep. And I stand by my statement. WAN != LAN. 10GBASE-T 10 Gbit/s Ethernet over cat-6A will give you many times that. >> happily manage 4-500MB/s machine to machine. > For hundreds of megabytes? I doubt it. What disks are you using that > are that fast? Yep. Stripes of mirrors of Seagate Constellation ES.3 drives. >>> replications. >> Google "incremental". > Google "restore from incremental backup" Old hat, we restore from snapshots. -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:00PM -0500 On 2/3/2016 6:07 PM, David Brown wrote: >> Oh, I know how to use them. No problem. > You have given every indication that you have no idea what version > control systems are for, how to use them, or what they can do. You have given every indication that you have no idea how to manage systems to limit failures. Something that is critical when dealing with more than one or two programmers. > First - no, repositories don't go bad except if you are /really/ unlucky > (such as too many disks in your raid dying at once). Or they go bad if > you use crap like MS Source Safe, which corrupts repositories. Repositories *can* and *do* go bad. Whether you're lucky or not, it happens. Not a problem for a one or two person development group. A major problem for 100+ developers. > And second, if it happens, then you have an IT problem, and the IT > department fixes it by restoring from backups. But that is not part of > the development process. The IT department is only responsible for restoring to the latest backup. Nothing more, nothing less. And you don't know for sure when that backup was. > (if you even have one) is in place again, copied over from one of the > other repositories. That is one of the points of having a distributed > version control system. Once again you show your extreme ignorance. git is far from the best, and is never used in bigger development groups. They use commercial systems which are much more reliable. But any version control system, no matter how distributed, can be corrupted. > whose claims are regularly so unbelievable and absurd that we cannot > accept you at your word). But if you don't believe me, then that's fine > - I care not. I don't care how you do backups. You just showed you have no idea what you're doing. And your reputation means nothing. You have shown zero knowledge of how large projects work. But then I have yet to see anyone with any experience in large projects posting here. Everyone seems to be be one or two person development groups. So you're not unique. > other copies. If developers are making their own backups and restoring > the repositories from them, then your system is more screwed than I had > previously imagined. No, developers don't. But the project manager and assistants do. That's true whether it's local or distributed. >> things happen. Just backing out a change does *not* work 100% of the >> time! > Yes it does, if you know how to use your tools. Not when the repository is corrupted, for instance. >> Day 2. Developer is working on different code. > While leaving half-done code outside of the version control system, and > with no idea whether it works at all? Welcome to chaos. Nope. He's done with that code and working on something entirely different. No chaos involved - the system's design shows what he needs to do. But you probably don't do designs, either. > 1.21.3" and "ready to test 3b". He then spends the afternoon begging > the department head to let them get a proper version control system, and > to let them use it. No more than trying to find the right file from any other repo. > the idiotic systems that meant it took 30 programmers to do the real > work of 2 people. > Perhaps your problem is that you have only worked for companies like that. Yes, perhaps my problem is that I've only worked for companies who know what they are doing - including IBM. You had a summer job interview. I had 13 years as an employee. And I hate to tell you - but the interviewer was being kind. They tell something like that to every one they turn down. They aren't going to tell you you aren't qualified. And 8 of those years was in software - where we worked on code most of the week. We had maybe two hours of meetings a week; most weeks we had less. But from your posts here, I would have told you exactly the same thing - either as an IBMer or a project manager. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:02PM +1300 Jerry Stuckle wrote: > You haven't? I have. In fact, I've been using incremental backups > since the mid 1980's with DB2. A database, but the concept is identical. You are still living there aren't you? > idea what you're talking about. To restore from snapshots, you need to > go back to the last full backup, then restore each snapshot - *in > order*. Nope. On modern filesystems, a snapshot is a full representation of the filesystem at the time it was taken. To restore, you just copy the files from the snapshot. -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:06PM -0500 On 2/3/2016 5:37 PM, Paavo Helde wrote: > to work meanwhile, later commits might just be more coarse-grained than > ideal (even this could be avoided by using a multi-stage VCS like git or > mercurial). Never does not mean can't. In 38 years of being employed in the computer field, I've seen similar things happen several times. The worst case was 9/11/2001, when the World Trade Center was destroyed. None of my customers were directly involved, but I know those of some consultant friends of mine who were. Some companies had off-site backups and were back running in a couple of days. Others had off-site backups of their data, but no backup systems and were out for weeks. None of them had offsite backups more than 36 hours old (the night before the attack), AFAIK > should set up our own server, using the copy of the repository which is > mirrored to our site every night. But I guess there would be no hurry in > such a situation. For a project with dozens of developers, every hour down can cost tens of thousands of dollars. Backups like you're taking are quite important, and keeping those backups at a different physical location critical. However, if you wait for IT to restore your repos, you might be down for days. If you have your own backups of the repos, you can rent a server and be back up in a few hours. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:09PM -0500 On 2/3/2016 6:08 PM, Ian Collins wrote: >>> team of developers using this repository. > Me neither. If your infrastructure is sound, you never will. If your > infrastructure isn't sound, get one that is. Just because you've never lost one doesn't mean it can't happen. > single > repository > There, is that clear? Yes. But that still doesn't mean the repository itself cannot be corrupted - even if it is distributed. You obviously have no idea what can happen, and has happened, in the real world. Big companies know the potential problems, and take steps to prevent them. This includes backups of the repositories, separate from system backups. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:12PM +1300 Jerry Stuckle wrote: > On 2/3/2016 6:07 PM, David Brown wrote: > Once again you show your extreme ignorance. git is far from the best, > and is never used in bigger development groups. Er the Linux kernel? The Illumos kernel? The... and so on. Seeing as you like them so much, read: http://www.ibm.com/developerworks/library/wa-git/ -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:14PM -0500 On 2/3/2016 5:03 PM, Geoff wrote: > What tools are you using in your work where you can even conceive of a > scenario where a development team with a version control system could > have a singular copy of a file? You are assuming that the RCS is recoverable. That is not always true. You fail to understand that *any* data can be corrupted - even a repo. The fact you've never had it happen means only that you've been lucky. It's not a matter of *if* something happens - it's a matter of *when* it happens. It's why large companies spend millions of dollars every year (some every week) maintaining backups. Using multiple systems limits the effect of hardware failures. But that's all it limits. Software failures and deliberate sabotage can destroy any system. If you think it won't happen, look at the facts - the vase majority of successful hacking attacks start with someone who has legitimate access to a system. It can be a disgruntled employee - or someone who was sloppy with a password. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:36PM -0500 On 2/3/2016 7:49 PM, Ian Collins wrote: >> full backup. That may have been weeks ago. > Bong! Wrong again. I restore from the appropriate snapshot. Each > snapshot represents the state of the filesystem when it was taken. Then it isn't a snapshot. It is a backup. Snapshots only store changes since the last backup or snapshot. So you really don't know what a snapshot is, do you? > get many megabytes of data every 15 minutes, but nearly all of that will > be changes to the same files, the the aggregate isn't much bigger the > the individual replications. Maybe, maybe not. But if it's a snapshot, it has *only* the changes since the last backup or snapshot. Nothing more. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:40PM -0500 On 2/3/2016 7:59 PM, Ian Collins wrote: >> Yep. And I stand by my statement. > WAN != LAN. 10GBASE-T 10 Gbit/s Ethernet over cat-6A will give you many > times that. I understand that. And what does your disk do? >> For hundreds of megabytes? I doubt it. What disks are you using that >> are that fast? > Yep. Stripes of mirrors of Seagate Constellation ES.3 drives. Maximum sustained transfer rate: 175MB/s. Actual rate will be less than 1/2 that in production. >>> Google "incremental". >> Google "restore from incremental backup" > Old hat, we restore from snapshots. Snapshots only contain the changes from the last backup or snapshot. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:41PM -0500 On 2/3/2016 8:02 PM, Ian Collins wrote: > Nope. On modern filesystems, a snapshot is a full representation of the > filesystem at the time it was taken. To restore, you just copy the > files from the snapshot. That is known as a backup, not a snapshot. Snapshots are incremental backups. And if it were the full file system - backing up a 100 MB repo every 15 minutes would be 9.6 TB per day. But you claim it isn't anywhere near that. Which is it? It's becoming more and more obvious you have no idea what you're talking about. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Feb 03 08:45PM -0500 On 2/3/2016 8:12 PM, Ian Collins wrote: > Er the Linux kernel? The Illumos kernel? The... and so on. > Seeing as you like them so much, read: > http://www.ibm.com/developerworks/library/wa-git/ Big companies, including IBM, don't use git internally - at least none of the projects I know of do, for a number of reasons. They use commercial and/or internally developed software. But it's free, for cheapskates who won't pay for a more robust commercial system. IBM would be stupid to ignore that fact. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:46PM +1300 Jerry Stuckle wrote: >> snapshot represents the state of the filesystem when it was taken. > Then it isn't a snapshot. It is a backup. Snapshots only store changes > since the last backup or snapshot. You really can't learn, can you? From the ZFS man page: Snapshots A snapshot is a read-only copy of a file system or volume. Snapshots can be created extremely quickly, and initially consume no additional space within the pool. As data within the active dataset changes, the snapshot consumes more data than would otherwise be shared with the active dataset. > So you really don't know what a snapshot is, do you? I know they weren't around in the 80s, but if you take the trouble to read up on modern filesystems you might gain some understanding. >> the individual replications. > Maybe, maybe not. But if it's a snapshot, it has *only* the changes > since the last backup or snapshot. Nothing more. The difference between any two snapshots is what an incremental replication sends. You are making an arse of yourself discussing technology you don't understand. -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:48PM +1300 Jerry Stuckle wrote: >> WAN != LAN. 10GBASE-T 10 Gbit/s Ethernet over cat-6A will give you many >> times that. > I understand that. And what does your disk do? So you did you say "you won't get 100MB/s throughput on a lan"? >> Yep. Stripes of mirrors of Seagate Constellation ES.3 drives. > Maximum sustained transfer rate: 175MB/s. Actual rate will be less than > 1/2 that in production. Google "stripe". >>> Google "restore from incremental backup" >> Old hat, we restore from snapshots. > Snapshots only contain the changes from the last backup or snapshot. Nope. -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:56PM +1300 Jerry Stuckle wrote: >> files from the snapshot. > That is known as a backup, not a snapshot. Snapshots are incremental > backups. You have no clue about modern filesystems, do you? "zfs snapshot <filesystem>@<snapshot name>" creates a snapshot. "zfs send -i <filesystem>@<snapshot one> <filesystem>@<snapshot two> creates an incremental stream comprising the differences between the two snapshots. A snapshot is not a backup, it is a snapshot (hence the name) in time of the filesystem. > And if it were the full file system - backing up a 100 MB repo every 15 > minutes would be 9.6 TB per day. But you claim it isn't anywhere near > that. Which is it? That depends on how much churn there is in the repository. > It's becoming more and more obvious you have no idea what you're talking > about. It's becoming more and more obvious that I am wasting my time trying to educate you. -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 04 02:59PM +1300 Jerry Stuckle wrote: > Big companies, including IBM, don't use git internally - at least none > of the projects I know of do, for a number of reasons. They use > commercial and/or internally developed software. They own Rational, so that isn't surprising. Who is one of the biggest contributors to the Linux kernel? IBM. -- Ian Collins |
| David Brown <david.brown@hesbynett.no>: Feb 04 09:56AM +0100 On 04/02/16 01:42, Jerry Stuckle wrote: >> producing a movie or running a particle accelerator, you don't make >> terabytes of /new/ data every day. > Backing up 100MB every 15 minutes comes out to about 9.6Tb per day. Yes, copying 100 MB every 15 minutes means 9.6 TB per 24 hours. So what? You picked a totally arbitrary number and extrapolated wildly. First, productive though he surely is, I don't expect Ian to produce 100 MB of change every 15 minutes. Even his whole team (however big that might be, I've no idea) will not produce 100 MB of code changes every 15 minutes. Per developer, the median will be a few KB at most. If this is the machine that stores everything, including generated object files, debug files, etc., then changes will be bigger. But as I said, I don't believe you know how snapshotting works. My guess is that Ian is running this on a ZFS filesystem, since he is a Solaris fan. I haven't used ZFS, but I use btrfs on my Linux systems, and the features are similar. A snapshot is done in a fraction of a second, regardless of the size of the change. Think about that for a moment - it will take time to sink in. When passing the data out to a different machine (on-site, off-site, whatever), then the data must pass along a wire. But the data that needs to be sent is not the sum of the changes since the last backup - most of the big changes will be multiple changes to the same file, and only the last change gets sent. Filessytems like ZFS and btrfs have tools to let you send over only the data that has changed since the last copy. > Copying the changes means you have to restore starting with the last > full backup. That may have been weeks ago. I don't know whether it is just /you/ that is stuck in the 1990's, or whether all your "Fortune 500" customers are also there. But it is a good long time since people have used tape systems with monthly full backups and daily incremental or differential backups. Now, I don't know the details of the backup system at Ian's workplace. But this is how things work at mine: If I have a file /home/david/test.txt, and I want to get the version from 01.06.2015. I can find it on my machine at /snaps/01.06.2015/home/david/test.txt. If my machine has crashed and lost everything, I can find it at the same place (roughly) on the backup server - it's a quick ssh away. If that has crashed too, I can find it at the same place on the off-site backup server - also a quick ssh away. On the server side, if a VM dies horribly, I can use a snapshot of the whole server for a restart. The snapshots are all there, ready to go. That's what you get with snapshotting, and proper backups. (And of course, development files are all in the version control repositories, and so old versions are available from there.) |
| David Brown <david.brown@hesbynett.no>: Feb 04 10:15AM +0100 On 04/02/16 02:48, Ian Collins wrote: >>> times that. >> I understand that. And what does your disk do? > So you did you say "you won't get 100MB/s throughput on a lan"? I don't think Jerry has heard of 10 Gb over copper. And I suspect he things WAN means ADSL or perhaps a T1 link - it seems that using fibre for WAN is exotic in his world. And remember, Jerry is used to IBM 350 disks - the idea that a bog standard cheapo hard disk can sustain well over 100 MB/s sequential transfer is science fiction to him. >> Maximum sustained transfer rate: 175MB/s. Actual rate will be less than >> 1/2 that in production. > Google "stripe". Also google for raid in general, disk caching, and SSD's (sit down before you read that one, Jerry). A machine capable of sustained random-access transfers of 500 MB/s total to a number of different clients can probably be built for about $4000-$5000. >>>> Google "restore from incremental backup" >>> Old hat, we restore from snapshots. >> Snapshots only contain the changes from the last backup or snapshot. For your convenience, Jerry: <https://en.wikipedia.org/wiki/Snapshot_%28computer_storage%29> |
| David Brown <david.brown@hesbynett.no>: Feb 04 10:23AM +0100 On 04/02/16 01:46, Jerry Stuckle wrote: > order*. One snapshot missing or corrupted? EVERYTHING after that is > lost. And if you try to restore later snapshots, you will have a > corrupted repo - one where you have no idea what is good and what is bad. That is not shapshots - that's is 1980's style incremental backups. It's a different world. Until you learn that, you are going to remain confused in this discussion. And just because I have never had to recover a repository from my backups, does not mean I don't know how to do it - I do, and have have tested restoring virtual machines from the snapshots. The time to learn about how to restore a VM (such as our version control server) is when creating a backup procedure, not when you /have/ to restore it. |
| "Öö Tiib" <ootiib@hot.ee>: Feb 03 11:11PM -0800 On Thursday, 4 February 2016 01:04:28 UTC+2, Mr Flibble wrote: > > The code tells you what it does. Nothing more. > Well written and designed code with sensible, descriptive variable, > function and class names is virtually self-documenting. Reading comprehension problem? Vir Campestris wrote above that code can't answer without comments to question why it does what it does and not something else that appears simpler. Can you demonstrate that it can answer to that question? There are no point to write those bald assertions. If maintaining developer lacks that answer in code then she can break some unit test naively and waste her time. If there are also no unit tests that demonstrate the reason why then that typically results with regression. |
| bleachbot <bleachbot@httrack.com>: Feb 04 01:15AM +0100 |
| Ramine <ramine@1.1>: Feb 03 07:15PM -0800 Al Salam alaykum... You have to believe in our beloved Jesus Christ. Jesus Raises Lazarus from the Dead Read the story here: http://christianity.about.com/od/biblestorysummaries/p/raisinglazarus.htm Thank you, Amine Moulay Ramdane. |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment