- Unit testing (Re: Putting project code in many different files) - 18 Updates
- TDD considered harmful - 7 Updates
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:25PM -0500 On 1/31/2016 9:00 AM, Jorgen Grahn wrote: > There is no reason it should be manual in small or medium-sized > projects ... building and running unit tests is one of the most easily > automated tasks I can think of, provided you have half-decent tools. It depends. I've seen a lot of small projects where the effort of automating the test exceeds the manual effort required to do the testing. > (Except if you're cross-compiling and this is the only thing you want > to run on the host. But then doing it manually would be even harder.) Maybe, maybe not. The tests would be the same, even when cross-compiling. And, in fact, cross-compilation is even more likely to be manually tested because the tools to do it automatically may not be available on the target system, system constraints may not allow for the tests to be run, or any of a number of other reasons. > - The output when everything is fine doesn't fill your screen with > information you won't read ("hey, did you know these 4711 tests > all passed -- again?" followed by a complete list). Tests should never be part of the build and should be performed by a separate group, with the tests designed based on the documentation. Programmers do a lot of building - you don't need to test with every build. And the people writing the code should never be part of the test environment - and aren't, in well-run projects. If the programmer writes the tests, he/she will have a tendency to write the tests based on the code they wrote (natural inclination). A separate group writing the test will not do that. > optional, but without all three bullets above, unit tests will never > work well (IMO, of course). > /Jorgen Builds may be nightly - not necessarily. And by the time you get far enough to do a nightly build, you should already have a lot of tests run. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:34PM +1300 Jerry Stuckle wrote: > The behavior of the real B's methods when used with A are the same in > testing as in production. If the test requires one of B's methods to > fail, then it will fail in both test and production. So how do you get one of B's methods to throw an out of memory exception? Or a <you name it> exception? Or return the result you expect it to return after 100 calls? -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:36PM +1300 Jerry Stuckle wrote: >> Which would be pretty f'ing useless. > Which is exactly why you wouldn't be using a mock of B. You'd use the > real, *tested* B. If a real B didn't behave as it's tests define it, it wouldn't be a tested B, would it? -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:37PM +1300 Jerry Stuckle wrote: > You cannot properly test a class if behavior of dependent classes is > mocked. You cannot ensure the dependent class's behavior will be the > same when the real dependency is used. If that is the case, your process is broken. -- Ian Collins |
| Paavo Helde <myfirstname@osa.pri.ee>: Feb 01 01:38AM +0200 On 1.02.2016 1:13, Jerry Stuckle wrote: > Alf, What I mean is that A is tested with the real B. If A depends on > B, you can't ensure A works properly if you don't have a working B to > test with. You can never ensure that something works 100% properly. At best you can hope to have a good enough test suite to catch most of the bugs. If a bug surfaces in B, then there is not really much difference if it is the unit test for A or for B which catches the problem. To have separate unit tests for A and B is just to reduce the cyclomatic complexity in writing the tests, there is no deep philosophical reason to separate them. In TDD, if you develop a feature provided by A, you write a test for the feature using A. The test first fails. Then you start to add code to make the test pass. During that you see that one of the reasons why the test fails is that B is not yet written. So you write B, adapt A to use it and make the test pass. There is nothing in TDD that says you have to figure out the class dependencies beforehand and start with lowest level classes. TDD does not care about such implementation details. To ensure better test coverage, you may later add a separate test for B, but this would then be a pure unit test, not a TDD test (as it does not cover a feature, but a technical detail). Cheers Paavo |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:42PM +1300 Jerry Stuckle wrote: > Tests should never be part of the build and should be performed by a > separate group, with the tests designed based on the documentation. *Unit* test can (and should if your process requires test first coding) be written and run by the developers. Integration, system and acceptance (black box) testing may be performed by others if the team include testers. -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:42PM -0500 On 1/31/2016 6:34 PM, Ian Collins wrote: > So how do you get one of B's methods to throw an out of memory > exception? Or a <you name it> exception? Or return the result you > expect it to return after 100 calls? The same way you would in production - you create the scenario where the exception would occur, and cause it. If it's impossible to test, then it's impossible for the scenario to occur in production, anyway. And if you expect a specific return after 100 calls, then you call it 100 times. Anything that can happen in production can be tested for. Whether you test for the extreme cases or not is up to you. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:49PM +1300 Jerry Stuckle wrote: > The same way you would in production - you create the scenario where the > exception would occur, and cause it. If it's impossible to test, then > it's impossible for the scenario to occur in production, anyway. So you consume all of the memory on your test box, what then? > And if you expect a specific return after 100 calls, then you call it > 100 times. ..and if each call would fetch and parse a file from the internet? > Anything that can happen in production can be tested for. Whether you > test for the extreme cases or not is up to you. Anything can be tested if you can afford the cost. Unit tests generally can't afford to trash the host they are running on or create a DOS attack on a remote server... -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:50PM -0500 On 1/31/2016 6:38 PM, Paavo Helde wrote: > unit test for A or for B which catches the problem. To have separate > unit tests for A and B is just to reduce the cyclomatic complexity in > writing the tests, there is no deep philosophical reason to separate them. That's very true. But there is a very good reason for testing separately - once you get rid of the bugs in the independent class, you can be pretty sure any bugs which show up are in the dependent class. If you don't test the independent class first, you have no idea where the problem lies. And as you get more and more classes involved the problem becomes more difficult. > feature, but a technical detail). > Cheers > Paavo The design will define the class dependencies. And every time you have to rewrite A to adapt it to be, you are wasting programmer time, test group time, and potentially inserting more bugs. A proper design means the independent classes can be written first, and the dependent ones later. This eliminates the need for programmers to rewrite dependent code. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:51PM -0500 On 1/31/2016 6:36 PM, Ian Collins wrote: >> real, *tested* B. > If a real B didn't behave as it's tests define it, it wouldn't be a > tested B, would it? No. But a mock B wouldn't be used, because it doesn't necessarily behave the same as the real B. If it did, it would be a real B. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:54PM -0500 On 1/31/2016 6:37 PM, Ian Collins wrote: >> mocked. You cannot ensure the dependent class's behavior will be the >> same when the real dependency is used. > If that is the case, your process is broken. If you think otherwise, then your process is broken. Say you get the dependent class working with a mock independent class. Now you create the independent class and test it. The dependent class now fails tests because the mock independent class didn't perform the proper actions. So now you have to go back and rewrite the dependent class to work with the real independent class instead of the mock one. You have now wasted programmer and test group time by having to rewrite "working" code, and introduced the possibility of even more bugs. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:55PM -0500 On 1/31/2016 6:42 PM, Ian Collins wrote: > be written and run by the developers. Integration, system and > acceptance (black box) testing may be performed by others if the team > include testers. Not a chance. Never in any of the major projects I've been involved in have any tests been performed by the developers. That's where the bugs come in, because the developer of a class may not have the same understanding of what the class should do (which is more common than you would think). Developers will tend to write tests to the code. Test groups will write tests to the documentation. Two entirely different things. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Ian Collins <ian-news@hotmail.com>: Feb 01 01:03PM +1300 Jerry Stuckle wrote: > Now you create the independent class and test it. The dependent class > now fails tests because the mock independent class didn't perform the > proper actions. So you didn't implement your mock correctly, go back and fix it. > the real independent class instead of the mock one. You have now wasted > programmer and test group time by having to rewrite "working" code, and > introduced the possibility of even more bugs. You have wasted time implementing a mock that didn't do what it said on the tin. -- Ian Collins |
| Ian Collins <ian-news@hotmail.com>: Feb 01 01:06PM +1300 Jerry Stuckle wrote: >> include testers. > Not a chance. Never in any of the major projects I've been involved in > have any tests been performed by the developers. There you go, on every project I've working on in the past decade and a bit we have been using TDD, so all of the unit tests have been written by the developers. All of the system and acceptance tests have been written by our testers. > would think). > Developers will tend to write tests to the code. Test groups will write > tests to the documentation. Two entirely different things. Which is why we write the tests first. -- Ian Collins |
| "Alf P. Steinbach" <alf.p.steinbach+usenet@gmail.com>: Feb 01 02:54AM +0100 On 2/1/2016 12:49 AM, Ian Collins wrote: >> exception would occur, and cause it. If it's impossible to test, then >> it's impossible for the scenario to occur in production, anyway. > So you consume all of the memory on your test box, what then? Well this was called "stress testing", some 15 years ago. Checking… Yep, still. https://en.wikipedia.org/wiki/Stress_testing_%28software%29 But yes, I agree, it would be impractical to ONLY wait for a rare but important condition to occur, so as to observe it. Better to force it. I guess that's the same as with an engine: the parts have to fit, but one doesn't use actual fitness as primary testing, because that would be impractical. So one measures. And only if the measurements say OK, does one then assemble the thing and test the complete reality. Yes? Cheers, - Alf |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 09:50PM -0500 On 1/31/2016 6:49 PM, Ian Collins wrote: >> exception would occur, and cause it. If it's impossible to test, then >> it's impossible for the scenario to occur in production, anyway. > So you consume all of the memory on your test box, what then? Then you test. >> And if you expect a specific return after 100 calls, then you call it >> 100 times. > ..and if each call would fetch and parse a file from the internet? Then it fetches and parses a file from the internet. > Anything can be tested if you can afford the cost. Unit tests generally > can't afford to trash the host they are running on or create a DOS > attack on a remote server... If a unit test trashes the host or creates a DOS attack on a remote server, then the test is incorrect or the class fails. If it can happen during testing, it can happen during production. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 09:52PM -0500 On 1/31/2016 7:03 PM, Ian Collins wrote: >> now fails tests because the mock independent class didn't perform the >> proper actions. > So you didn't implement your mock correctly, go back and fix it. And how do you know if it is implemented correctly if it isn't the full class? >> introduced the possibility of even more bugs. > You have wasted time implementing a mock that didn't do what it said on > the tin. No, you have wasted time implementing a mock which doesn't perform the same as the real unit. Tell me - how is your mock class going to do the same work as the real class if it isn't the real class? It's a lot more than just returning a value from a method call. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 09:53PM -0500 On 1/31/2016 7:06 PM, Ian Collins wrote: > bit we have been using TDD, so all of the unit tests have been written > by the developers. All of the system and acceptance tests have been > written by our testers. Too bad. I'm sure you had a fair amount of bugs in the code. >> Developers will tend to write tests to the code. Test groups will write >> tests to the documentation. Two entirely different things. > Which is why we write the tests first. Which is exactly why developers do not write the tests. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:29PM -0500 On 1/31/2016 1:55 PM, 4ndre4 wrote: > you can build with them. Remember that there are languages (such as > Python, JavaScript, etc..) where there is no concept of "private > method". Private methods are just a convention. If the circuit works properly, you don't have to test each individual electronic component. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 06:38PM -0500 On 1/31/2016 6:06 PM, Mr Flibble wrote: > testable public methods. TDD is the enemy of both encapsulation in > general and C++ in particular. > /Flibble Yes and no. Private methods are untestable directly, but they can be tested through the public methods. If the public methods don't call them, then the private methods are unusable anyway. And even whether a private method exists or not is immaterial; one could easily add or delete private methods and change the public methods appropriately without changing the operation of the class. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Jan 31 11:45PM On 31/01/2016 23:38, Jerry Stuckle wrote: > immaterial; one could easily add or delete private methods and change > the public methods appropriately without changing the operation of the > class. You obviously don't understand TDD then and its mantra of designing upward from failing tests. Private methods are totally at odds with TDD. /Flibble |
| Ian Collins <ian-news@hotmail.com>: Feb 01 12:51PM +1300 Mr Flibble wrote: > You obviously don't understand TDD then and its mantra of designing > upward from failing tests. Private methods are totally at odds with TDD. Repeatedly burning your sausages doesn't make them good sausages. -- Ian Collins |
| Mr Flibble <flibbleREMOVETHISBIT@i42.co.uk>: Feb 01 12:56AM On 31/01/2016 23:51, Ian Collins wrote: >> You obviously don't understand TDD then and its mantra of designing >> upward from failing tests. Private methods are totally at odds with TDD. > Repeatedly burning your sausages doesn't make them good sausages. Except if you use TDD you don't end up with sausages you end up with meatballs. TDD is anathema to best (SOLID) practices related to the design of non-trivial software systems. /Flibble |
| Ian Collins <ian-news@hotmail.com>: Feb 01 02:13PM +1300 Mr Flibble wrote: > Except if you use TDD you don't end up with sausages you end up with > meatballs. TDD is anathema to best (SOLID) practices related to the > design of non-trivial software systems. Restating the same empty nonsense won't make it any more correct. There aren't any conflicts between TDD and SOLID practices. SOLID practices help make TDD better. If I recall correctly, SOLID was originally coined by Robert Martin (Uncle Bob) (possibly here: http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod). You may be surprised to know that he is also a strong advocate of TDD: https://blog.8thlight.com/uncle-bob/2014/05/02/ProfessionalismAndTDD.html Go figure. -- Ian Collins |
| Jerry Stuckle <jstucklex@attglobal.net>: Jan 31 09:48PM -0500 On 1/31/2016 6:45 PM, Mr Flibble wrote: > You obviously don't understand TDD then and its mantra of designing > upward from failing tests. Private methods are totally at odds with TDD. > /Flibble Oh, I understand it. And private methods have absolutely nothing to do with TDD. TDD deals with the external (public methods) interface of the class. The internal (private methods) of the class are immaterial. In fact, the private methods can change, be added or deleted without changing the public interface - which is what TDD cares about. How something is done - the implementation (which includes private methods) is immaterial. What is material is the results. -- ================== Remove the "x" from my email address Jerry Stuckle jstucklex@attglobal.net ================== |
| You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page. To unsubscribe from this group and stop receiving emails from it send an email to comp.lang.c+++unsubscribe@googlegroups.com. |
No comments:
Post a Comment