tag:blogger.com,1999:blog-10748614.post4966558563058294656..comments2024-02-28T12:34:09.691-05:00Comments on FrazzledDad: 31 Days of Testing—Day 6: Types of Automated TestingJim Holmeshttp://www.blogger.com/profile/05869146736565695900noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-10748614.post-45317234555263220112011-12-13T22:52:21.118-05:002011-12-13T22:52:21.118-05:00I was going to ask if you distinguished between fu...I was going to ask if you distinguished between functional tests and UI/GUI tests. Based on the chat with Roman, you apparently keep them in the same family.<br /><br />On a large client project, we did no GUI testing (didn't have a viable solution to test through the UI), but did have a large functional test suite that hit all the functions just below the UI. With very little logic in the UI, we covered a lot of our functionality this way, and had a very robust set of tests hammering away at the whole app. (Minus the UI, of course.)<br /><br />In a prior post you mentioned that "you can't test it all" which is exactly what led us to that point. The cost of getting GUI tests in place was going to far outweigh their value in that project.<br /><br />One other comment on this post, I'm a HUGE fan of isolation frameworks for unit testing. However, I've seen a lot of people start with the isolation framework and not really understand what's going on "under the hood." It's usually well worth the time to step back with the folks new to mocking/stubbing and show them a few hand rolled mocks/stubs using interfaces in a static language, or some class over riding in a dynamic language. There's a whole heck of a lot of "magic" going on that's pretty easy to explain when you take the time. (Always take the time! :) )<br /><br />Great series so far, Jim.Timhttps://www.blogger.com/profile/11470576015704065027noreply@blogger.comtag:blogger.com,1999:blog-10748614.post-27395568252854590102011-12-07T13:00:19.135-05:002011-12-07T13:00:19.135-05:00@Jim Holmes: I do not disagree. 95% of the applica...@Jim Holmes: I do not disagree. 95% of the application is in that "core" part covered by acceptance test (as I define it). I agree that end-to-end automated testing is required as well, but such end-to-end testing only needs to test your connection assumptions. End-to-end testing is what I would call integration test. It verifies connection of your core-logic with external components. Where external components are: disk, web, db, UI, clock. The big point is that UI is conceptually no different from web access or reading a system clock from integration test stand point. Hexagonal architecture is a very good demonstration of this concept. By lumping all integration points into end-to-end test suit you have only one group of slow tests. <br />But as always in this discipline - "it depends".Roman V. Gavrilovhttps://www.blogger.com/profile/11579626284365390077noreply@blogger.comtag:blogger.com,1999:blog-10748614.post-85212338991669718712011-12-07T12:19:32.709-05:002011-12-07T12:19:32.709-05:00@Roman: I think that's an interesting approach...@Roman: I think that's an interesting approach, but I'd disagree that you're truly checking functionality. You've said you're stubbing out various components, so there's no way to validate they function as expected. Moreover, you're below the UI layer, so you're missing critical aspects of that as well.<br /><br />Don't get me wrong: I agree there's value with an approach like that, but I wouldn't call it "Acceptance" or say it checks functionality. You're missing too much there, IMO.Jim Holmeshttps://www.blogger.com/profile/05869146736565695900noreply@blogger.comtag:blogger.com,1999:blog-10748614.post-11295994267103879832011-12-07T11:53:18.360-05:002011-12-07T11:53:18.360-05:00I would also add a type of a test, that I call Acc...I would also add a type of a test, that I call Acceptance Test that works under the UI but above the services. This test layer addresses the expected functionality of the system, and still fast (because you stub out file system, web, database, system clock, etc.), therefore all you test is the function of your application (which should be defined by acceptance criteria, hence the name).Roman V. Gavrilovhttps://www.blogger.com/profile/11579626284365390077noreply@blogger.com