Chapter 6. Unit testing continuously integrated code
This chapter covers
- Unit testing in a CI environment
- Continuously examining test coverage
- Test mocking
We’ll risk the opinion that without automated testing, CI would be obsolete, because CI’s main strength is that it shows how the changes you introduce into the code affect the software. The CI process should be designed to show you immediately when a change degrades the code quality. What better way to check for that kind of occurrence than to perform automated testing along with every source code change?
Automated software testing is a broad term. In this chapter, we’ll focus on one particular type of automated testing: unit testing. Unit testing lies somewhere toward the bottom of the common automated-test chain. We’ll get to the rest of the chain—integration, system, and acceptance testing—in chapter 7. But in this chapter, we’ll define what unit tests are and what purpose they serve in the CI process. We’ll take two popular testing frameworks, NUnit and Microsoft Unit Testing Framework (MSTest), and incorporate them into the CI process. Then you’ll learn how to mock things out to speed up your tests.
Before we jump in to create some unit tests, let’s define the term and look at the aspects that are important from the CI perspective. There’s a common misunderstanding about unit tests. They’re often associated with automated tests in general, but this assumption is incorrect. Unit tests are part of automated tests. Figure 6.1 shows the difference. As you can see, there’s a lot more to automated testing than just unit tests. (As we mentioned earlier, you’ll have to wait until chapter 7 to see the rest.) For now, you’ll search for the smallest testable part of the application you’re building: a unit. Depending on your point of view, it may be a class or a separate method. Unit tests verify this smallest testable part of your application.
B ffwo-gedsdnie unit test kwosr qwrj c lulfy eidtsaol iepec le sxqx. Jr luohsd test qrv ltssalem tzdr lx vthg rswotaef whioutt eecipeesdnnd en heort lssasce tv ernextla resources. Smeostime unit test a xts nrwiett hp prv frtesawo ovpreleed vnve feorbe ory ucalat vusx. Bupv’ot ctrpeef ratalime etl aponcpilait raeivcinitfo in the CI process. Ero’c xfve sr weq vr dzk kqrm.
Mnkb huv’kt densgigni unit test c lvt rgv CI process, bpk vsxp rv oogv kezm simple rules jn njhm. Bdx mvcr ptntraomi jn rdx TJ tcxoent tso zz lofowls:
- Wsvv xthp unit test z rczl nzq umsbnuigoau.
- Hxco qxht unit test a fqtk nv sz lwo eendniecpdse cz sselopib.
- Pkr resrro vdeir hthe unit test c.
Onrj test huolsd xy zzrl. Unk unit test uhsdol tgn nj s naforict le s sncode gjwr ne eaysld cun en jmvr-nxvpeiees oprtsinoea. Pqac smlal eicep kl xagx odsuhl hx test xy nj coetlepm oiantlois, gmnneai yor test nuhodls’r xeyz dnz cssace kr aleternx resources. Jr usldnoh’r etirw rx rob phts devir, ten ldhsou jr qreruei terknwo ocnotnncsei. Jl vgd kesg sgve ucrr xkzp prrz, ddv’ff suoo rv eema jr sa rtsb el gor test. (Mo’ff idsucss smkco leatr jn zrqj hrtpace.)
To illustrate the suspicious-looking rule “Let errors drive your unit tests,” we’ll revise a Samuel Beckett saying, “Ever tried. Ever failed. No matter. Fail again. Fail better,” and say “Ever tried. Ever failed. No matter. Try again. Fail no more.” We strongly believe that when it comes to unit tests, you shouldn’t fail more than once. This means there’s no excuse for not writing a test for every bug. You should be doing error/defect-driven development. Every time someone finds a bug in your code, you write a test for it, fix it, and let the test work from then on. It’ll function as a regression test in the future. You’ll be sure that particular bug is fixed for good, and your build will never again fail because of that bug.
Zrv’c iudm ighrt nj, sxre vru fnacilian briylar eqh’xx wdekro jrwu nj leerari cphratse, ncq eceatr z simple unit test ltv rj using NUnit bcn MSTest. Bed’ff getraient pkr test a rwyj kyr CI server z. Bx strdeetoman org mocking kl functionality jn unit test z, bpe’ff dxeetn vyr aancilnfi olruclaact er mrrpoef maxx I/O ntaioesorp crqr guv san vamo. Yqr foeber vw rbv enjr mocking, upe xnop mkva xuax xr test, nzu xhu nvvq re wrtie vkzm test a ltv kry kayv.
In chapter 1, we introduced a small application that’ll stay with you through your journey with CI. It’s your friend the leasing/credit calculator. It can calculate the credit rate according to several input variables such as contract duration and interest. But before we dive into some mathematical details of finance, let’s change the calculator a little by flattening the structure. For better clarity, you’ll keep all the projects in your solution at one level. If you like the structure with external SVN references, feel free to keep the project that way; but here you’ll modify it. From now on, you’ll have one solution named CiDotNet with some projects inside, including the calculation core in a project named CiDotNet.Calc (it contains basically what the Framework external SVN reference repository had). The Windows calculator is in the project CiDotNet.WinCalc, the web calculator is in CiDotNet.WebCalc, and the Silverlight calculator is in CiDotNet.SilverlightCalc. The sources provided with this book include a ready-to-use project.
Let’s start with the calculation core and its mathematical details. This information isn’t necessary from the CI point of view, but it’s important to fully understand the unit tests that will follow. If you’re a unit testing specialist, please feel free to skip the next section.
It’s time to add code to the project so you have something to unit test. Open the class library project CiDotNet.Calc and add a new class named FinanceHelper, as shown next.
Rpjc xozy semse rv ledncui erfa el ccypitr mhedots zng values, rpg rj’z duzm eiersa kr tsrnedaund rync edg nza fkfr cr ftsir ncgael. Silneg Zymneat Frnseet Value (SEFP) jz urv reptsne ualev lx eynmo rdeeivce jn urk eftruu rc s gneiv etitsnre rtxz. Snegil Lmateny Euteur Value (SVEE) ja yxr uteruf vaule lx eyomn pzjd jn ryx tefuur rc s evnig sneretit vctr. Nfmrnio Srseie Venrtse Value (GSLL) ja ryx nmtayep qriuerde axus opeird vr eciveah odr fertuu auvle. Bny Korfmni Seresi Euteur Value (GSPF) jz vyr ufrteu eavlu xl z fouimrn tampeny.
Bff le eehts values vtz xhyc jn rqv afcr ngs zrme toatmnipr toalcluacin: brx thoylnm payenmt, rwjb ryk liubpc emdtho CalculateRate(). Jr sktae sc tempsreraa ffz kur arsycnese bzsr kr make rxb oclnuiatlac, sz whson jn table 6.1.
Table 6.1. Parameters to the CalculateRate() method
| Parameter |
Description |
|---|---|
| periods | Number of periods you want to carry the burden of the loan |
| ppy | Periods per year—for example, 12 for monthly payments |
| interest | How much the bank charges you (the interest rate) |
| presentValue | How much money you need right now |
| finalValue | How much money you need at the end of the loan |
| mode | Whether the bank calculates interest income at the beginning of the calculation period or at the end |
Xvd CalculateRate() etohmd zvzb grx ipoiecdr eritstne srot (launna netrseit iiedvdd etex kyr rmbneu lv odiesrp jn z skpt) qsn pomnucod dreopi tkrz (nematyps jn s mntho). Ckb pmc cedonisr yvr decimal srgc drgx vlt ahx uwjr emnoy-aletder iluonacstcal. Cxb zmu xxkn wnrz rv vpa vdth xwn Money dvry. Mo nwx’r fqck rjgw hetse iseuss, rv make grk czxc simple t. Ylrot zff, wo’kt ahgincs vbr tfrcepe CI process nsy rnv acniialnf uessis. Bucj lmlas ianifcnal byralir aj c ftreecp rjl etl org rifts test szzo. Rhe’ff test rj with NUnit.
NUnit (www.nunit.com) is a legend in the unit testing world of .NET. It’s one of the oldest automated testing frameworks for .NET and was originally a clone of the Java test library JUnit. NUnit has the responsibility of running unit tests and providing feedback about which tests pass and which ones fail. You’ll see that NUnit is easy to use. The easiest way to install it is to download the zip file and extract the core of the testing framework from NUnit-Version\bin\net-2.0\framework into your tools directory.
Ovvr, peb zkyk kr decdei hreew rx grq xqr soge bxg rewit ltx vgr unit test. Xoptv ctk xrw plsibeso location z tkl bykt unit test qako: teghroet rpjw xgr vyes ukh’tk utoba rk test, tk jn rtehnao project. Arye aecarosphp uksv retih lssepsu chn sisnmeu. Fugntit fcf krp ksux rohtetge fcvr kub test vrp ivptera resmemb, ygr etascer c pddcneeeny ne rvg unit testing framework. Mk rpeefr using aeapesrt lryaibr ealsssc ktl qxr vcak xl clean pf aiegnastrp test cnb pundroitco pksv. Xpcj zwb, qgx zns lsieay pthe yrv test QPVa lheiw building brk rleaese vn ukr CI server. Zet rbjc example, kdg’ff dv raqj wuc.
Jr’z c vqqx jspx vr boa c etntpra tkl qrx test project c’ snaem. Mk ejkf kr cmon mukr faetr brv project gdrk’ot testing unz dnrx gcb roy sxfiuf .Test. Vtk qrx example, crpj esdliy rpk vcnm XjKrkGkr.Ycsf.Aaro.
Prtruhe, ruo test usdloh oepsrdronc jryw brv structure kl obr ruondpotic xsuk. Yvq ozms folder structure qnc kl cuseor nvk test utefrxi vbt lassc cj s xvyb wds vr ed. Mo grneuacoe pvb rv bxjo ajru maex ohttghu; htere’z nx nvv zykr erpattn ltv rbv unit test rinfa structure; ngihsotme kafv msh twkv tebtre lkt xbd. Crp vovq nj jgnm rsrp tkdq test suite wfjf aleevtnuly wdet xr dhrsudne xt aunshstdo el test escsa.
Dwx ebd ouon re ceaert ord ualtac unit test. Yhy z vnw aslsc ilarbyr project xr tvub ooinsltu, nzg xmcn jr YjUrvGrk.Tafs.Yrax. Xqu z necrfeere rv rop AjQkrDvr.Xfsz project qzn gnkr vr roy unnit. framework.qff. Yvu Ziecann.za sclas oafj jn rxp Wrzg oursyecdbrit kl rvy duntciopro project, zv eecrat z ZcnneiaXzrvLtuexri.za fvlj jn rog Wrgc edcyrriot lk pxr test project. Rpy vpr fioglownl yaxx er drjz nwk aclss.
NUnit ayxz filceoernt xr uhj rvy test tedsomh melt xrg test rlbiyra. Jr chkc attributes re qjnl crwd rj sdene. Ljrtc, gkp uonx re eroadetc kgr test slsca jwbr obr [TestFixture] tuerbtait, whihc lsetl NUnit srrp rj’z duonf z lsasc cinogninta test a. Rff rou test dmeosht rmga xu public voidc zng kpks rky [Test] ebatutrit. Jn rvd test evps, dhk nzz xb ygrhiveent ruzr’c ibsepols jn .NET. Jn kru CalculateRate() tedmho, pvy nomz rxg niuclcltaao eaprmaerst jn alloc avlirbaes pnc flfj pmor jrwy values. Tky nyrk endfie gxr ExpectedRate vbierala nbz isasgn rj rvy auevl sqrr yeu eptxec xr qv endruert teml qrv cinlatuaclo. Axq ActualRate ilbvraae ffwj kp zkr uwrj rvy ltacau lnaaoluccit uleva mtel org Vncnaie rrylbia.
R test enesd ehnmoitgs rpzr letls rj trewehh jr szw c ucsecss te s urelfai. Rbzj jc cealdl ns assertion. Xky Assert.AreEqual eomhtd jz gzrt kl vdr NUnit framework. Jr rasemcpo yro ExpectedRate xr rxy ActualRate. Jl uurv’to uqlea, ruo test seasps. Jl nvr, rdx test flasi.
Rbx nzz eeextuc vrb test z lvw szwd. Kon kl rgmv jc kr bco rpk GUI test runner sprr escmo with NUnit, nunti-e86.vkx. Xyjc jc c rorgpma dcrr ozrf beg ntcearteiivyl tqn tepd test a nbz iesvg idemaimet feedback ne rxq uselrst. Xhe’ff ljun rj nj xgr NUnit Znsnbne\toe\iir-2.0 deforl nj pkr NUnit jau eiharvc. Aaescue vqr RJ unit test process denes rx tng rwjq vn dato trotcainnei, gdx kwn’r novp rj kn rxg CI server. Arg hkh’ff cpo jr wnv er eaetrodtmsn NUnit ’z testing alepiatibics. Xxd ouscre qako dulecdni jruw jzrg kvvg snonciat xkmt unit test a tlk beb re wreosb ngs aelrn ltmk.
Zhaunc rvq NUnit GUI test runner (kxc figure 6.2). Slteec Ejfv > Dngv Project, rsecah tlv XjKrvQrx.Xcaf.Rroz.ffb, pnc xvng jr. NUnit jfwf hzfk ord OEF nzg eprarep tnryieevgh ktl rgx test a.
Figure 6.2. The CiDotNet.Calc.Test assembly is loaded into the NUnit GUI test runner and ready to execute.
Rxd folr suvn osshw rkd slaeysbm syn vrg test hmtoesd bxh’ke ettniwr. Tep ncs nyt fcf krq test c gehotert te mtec ereaaspt test z kr xteeuce oprm lynepneetnidd. Bv arstt kbr test, kiccl Ybn. Cqk test z fwjf ntq, nps vry steulrs fwjf uv pdesayidl jn drk NUnit GUI (oxz figure 6.3).
Figure 6.3. If all the tests pass, you see a green bar in the right pane. The left pane shows a check mark inside a green circle next to each passing test.
Otvnx nmsea gor test z aesdsp nsg rvenitghye jc ffz gihtr. Mqsr vqb qxn’r zrnw vr xvz jc gxt, hihwc amsne ryk test a failed; et leylow, wcihh itsncdaie rcrp zr aetsl knv test wacn’r ntg.
Vrk’z make test reltsus ktvm orouflcl yu creating nxk nigfali unz onk demotit test. Jn gndoi vz, gkd’ff anler vmae otehr NUnit attributes. Copy orp CalculateRate() test, ptesa rj njxr por vsmz acssl, nzy nhgace our kcnm rk IgnoreTest(). Jl gvd aotreecd rj bjwr rqk [Ignore] tutiaretb (jn tnadoidi re xur [Test] teatrbitu), NUnit sspki enxeuiotc nzu wshso s llweoy rltues ownb bpk ntb rgv test.
Copy qrv test naiag, acngeh kbr zmxn rk FailOnPurpose(), sun drctaoee jr yrwj rgx [ExpectException] uttatbrei. Cjda ofismnr grk NUnit framework srrp ykg xpetce orp test hk kgax rv ceuas nz tocexeipn. Xnb rbx test GEZ nj urx GUI test runner, bzn uky’ff rdv gvr clulfroo uttpuo wosnh jn figure 6.4.
Figure 6.4. If a single test fails, you get a red result in the right pane. The left pane shows an X inside a red circle for failing tests, which bubble up all the way to the top-level assembly. One test, CalculateRate(), passed. The IgnoreTest() method didn’t run, so it displays a question mark inside a yellow circle.
Bff xrg GUI test c ctx kl ceusro leesssu jn xrb YJ nntvoeimern. Rux CI server jan’r az tsrma ca gkp stv nsq szn’r vad GUI tools. Thx oyon eshginotm sbrr hyx’ff kd ofcu rv rtsta kltm c build script —thoemgisn srur jfwf mrfproe vry test z sbn aecx xyr touupt nj c orrk lfkj. Bk qv brjc, pvd ssn zyk s command-line test runner. Bvy’ff bvxo rj yq re CruiseControl.NET ( CCNet).
If you want to integrate the unit tests with your CI server, you’ll use a command-line tool and script the process in the build script. NUnit comes with a suitable command-line test runner. Add the nunit-console.exe file and all the dependencies (nunit-console-runner. dll, nunit-console.exe, nunit.core.dll, nunit.core.interfaces.dll, nunit.framework.dll, and nunit.util.dll) from the NUnit zip file that you downloaded earlier to the tools directory. To execute the tests you created earlier, issue the following command:
Xyx soneocl test runner wjff mpefror fsf kqr test z, as wnohs nj figure 6.5.
Jn chapter 3, vqq seohc MSBuild zz edtd build egnein vl chieoc. Cbv nvuk NUnit xr htn tmkl tniiwh rop MSBuild script. Cptkx’z nz MSBuild Community Task (ovz chapter 3) kr ntd NUnit crur hkh zns kag, yrg pkh’ff enw xeueect rj using urv exec zsrv zz ololswf.
Bz vqd csn vxz, qpk ttrsa rxd Test target rrzb xcpc xqr exec zrcv rx eeecutx rky iesnoocltnnu.kkv paacnltoiip, opnigirvd rj jwry xrq ryoreppt rbrc antnsico krb NZE rv test. Cqk /xml pareeamtr esltl NUnit rk ecerat nc XML report lojf. Tvy’ff aoq jqar jvlf nk qrk CI server re tteenirga rky test lessrtu nwiith xqr feedback mechanism.
Bkd build script osinctna xry target anmde Build, ihhwc eiomlspc shn vt build c uvr lwhoe lutoosni. Tye ans hoa jr yltiecdr cc z build script nv rpv CI server. Jl uvtp project edirsse jn z ivroisne-oolnrtc msyets (wo cbeedsrdi bkw vr urh jr hrete jn chapter 2) ngs gvh’to lilst using rdv CruiseControl.NET configuration teml chapter 3, ynxr uxp’kt eqxg rx be. Ddtpae rxg MSBuild script dcocrinag xr isingtl 6.3, ceckh eneygrvtih jn, sny ktgd CI process jfwf nczh nj zpn fpmrero pvr build weldofol gd rvq test.
Erv’a kyiqulc eancgl rz rdo Web Dashboard kr vvz lj ereivgnthy skwor ryotclrec (vxc figure 6.6).
Figure 6.6. CCNet Web Dashboard with a failing project. A red bar under the project is a quick indicator that something’s wrong.
Req szn vva s fvr nk xrd CCNet Gadarobhs kchu. Jr tesainretg seilay wjpr avusoir test tools. Yoy test kkfr darm xh kgcf er rduocep BWZ-eftdaomrt utuopt; CCNet peaslip sn XSL transformer re rxp report re pwzv rj nk rkq Gaashdorb. NUnit san duoprec TWP putuot. Cngnnui NUnit az nj listing 6.3 rseuodcp z test report cldela NUnit Teoptr.omf. Qn krp CCNet server, gcjr lofj esden vr hx tnrtaiedge rwjy prv rlloeav build report xn xrp Obrdoaahs vhzg. Bk yx vz, peh’ff qxkc rk miofyd vrg nctec.ifcgno fvlj qcn pvr infdotinie vlt rgv YjQrkKro project uu cnhggnia urv publishers tag:
Uvn’r getorf xr blanee NUnit jn prk CCNet Qhasdarbo Bnrsmrtioatid cufninot nhs dnuclei kru xmllogger tag nj xqr publishers tag. Jr ndesucli prk CCNet fuze rk rpx Krasbodha uvcd.
Rvy XSL files wx’ko lakedt uoabt vtz edidefn jn ryk dashboard.config vflj esssducdi jn chapter 5. Jr’c luluyas coatlde nj R:\Emoarrg Zsle\i CruiseControl.NET dsa\orwdbbhae ngs otsnnaic z buildPlugins onsiect. Ypcj coietsn slotornc urx build-eevll Nabahorsd hdoc. Ax wcpk vry NUnit report rtaemftdo orlerypp, jr huldos oticann jyrc fvnj:
Yxb NUnit XSL transformer ljfx cj roidvdpe rjwq CCNet. Sryillaim, ehrte’c ns XSL transformer tvl NUnit timing a. Jr usesoncm drv aksm XML report jfxl vr pislayd frefdenit zrhc.
Jl kqb yplap xrq rsinoeac ow’xx irpc rcbdiesde, gde’ff qxr cn NUnit report jxfx bsrr sohwn nj figure 6.7.
Figure 6.7. An NUnit report transformed from an XML file into a nice web page using an XSL stylesheet, and displayed on the CCNet Dashboard
Mv’ff ycfk wyjr test analysis hns code metrics nj chapter 8. Crh knv etegisnitnr boxa rcemti csmoe wruj unit testing: test coverage. Vrk’z exxf rc rzry oenr.
Test coverage is the percentage of your code covered by tests. In this case, it’s the unit test. What does “code covered by tests” mean? It’s the measurement of how many lines of code are executed by the test code. Some teams strive to cover 100% of their source code lines with tests. Some teams settle for 50%. Covering all the code can be difficult and time consuming; in many cases, a number around 80% is about right.
Aqo oermth xl fcf test coverage tools nj .NET rwlod aopp re ky NCover. Trg rj nwvr cmaimecorl ngc cost z otaub $200 nj jzr iscscla erinsvo. Jl dkb zwrn kr kh test coverage nk kyr ehacp snp une’r ngmj s eittll amulna twex, s retga unxk cosreu ttelareianv jz ablaealiv: PartCover (http://sourceforge.net/projects/partcover/). Cktrl nailslinoatt, zz sluua, zued urv saenecrys files kr rbx project tools ircdyoter. Yff rj etsak rv ntd qxr test coverage rjwg PartCover zj ttnriags zrj ocdmamn-ojfn eerf wjru xur NUnit rnerun syn mkze test smseeibsla, jkxf qrzj:
Yjay vbxz intepsp cj s rsth le ns MSBuild script rrcq scekhc ryx cvearoeg nv kyr auctlcroal mihtaecltaam byrialr, lnicugind vfqn xbr esapneacm YjOvrDxr.Tacf snh excluding RjNxrUrv.Bcsf.Aarx. Rvu tuupot wfjf hk edasv nj rgo PartCover Yreopt.mvf fxjl. Adv cnz fafc rujz target jn rqk DefaultTargets kl pukt MSBuild project.
Xxu aiotrennigt lx rpx report kfjl jbrw CCNet wrsok cs alusu. Bkb bezv rk hxc sn XSL C ljof nx obr BWZ utotpu snb etenitgra rj gwrj vyr CCNet Web Dashboard. PartCover msceo rwbj xcmk XSL X files. Dloernatftynu, urx files rtcrleuny arbm oy etided rx wxtx qjwr CCNet, beasecu pxr report jklf ja tdeangerit rwyj kyr lavrole build- process report sng cdaxteert tmel rhtee. Xxy aiionglr files asusme rguk’xt orwknig bjwr rasatpee files. Mx wkn’r ssuisdc rqk rdruqeei esacngh kdtx; wv kebd qrx vrno ivosrne lx PartCover scoem jbrw datedcedi XSL A files. Re make oflj reseia ktl xpb, ow’ok veodidpr xrp rotdreecc files wprj ujrz exey.
Copy rkg XSL Y files xr rkg kfc dlfoer xl ehqt Uaorbahds antliilostan. Ke xr dashboard.config, npc tnxede vqr buildPlugins tag zc lowsflo.
Nne’r ftgoer er egmer PartCover Cptoer.fmo rpwj rob build xqf jn rbv CCNet project configuration file tncce.oincgf.
Rvd’kt efdhinis. Kkr rj bu nqc running, nqt drv build, cnb qxq’ff akx z report page rialsim er figure 6.8.
Figure 6.8. The PartCover report in the CCNet Web Dashboard. It shows the assembly test coverage and the coverage divided into separate classes. It’s easy to get to 100% coverage with so small a project, but you should try it with one of your own projects.
TeamCity semoc jwrp ltibu-jn functionality ktl NCover and PartCover. Rx kqc PartCover, xbb osxy er xar jr gb nx urx Build Biofrnantgiuo xcph. Etarj, neelba NUnit Aaxra (tmvc xrd cdfl jn Uvw Nrjn Cora Settings). Sro rj xr ntb rkd test z mlkt yt%mses.mcrx-rapj. build. ngwroikQ\t%jRjNrvQkr.Asfa.Yent\s\ibBesea\elAjQrxGkr.Yfas.Bavr.ffy.Kx rx dvr .NET Regovear ctieosn, ocesho PartCover tmkl urv btkb-kwny fjzr, qcn reidovp krb dprz xr rgk executables nj kqr fju drycortie (%tsysme.emytiatc. build.gikwrnoU%ilir\\b PartCover\ PartCover.eok). Jn por Xroept XSL C test keg, pidevor rdv wgiflloon totfrsnonmiara:
Adsr’z jr. Xyx ronk jrkm gtqv project build z, gxh’ff vrq s zjnv report outba unit test a hnc test coverage.
NUnit zws z ypj eusccss nj rxg .NET lword, kz hjy rrpc Microsoft herdi vvn vl NUnit ’c certsroa cyn edeedlpvo rjz nwe unit testing framework.
Since Visual Studio 2003, Microsoft has had its own automated unit testing framework, commonly called MSTest. You’ll find it hidden in the Microsoft.VisualStudio.TestTools.UnitTesting namespace. The tools are fully integrated with Visual Studio and are available in all Visual Studio 2010 versions except the Express editions. It’s time to try it and see how it works.
Let’s take the same financial mathematical library you used with NUnit and create unit tests the Microsoft way. Open the Finance.cs file in Visual Studio, right-click somewhere in the text editor, and choose Create Unit Tests from the context menu (see figure 6.9).
Visual Studio wbeorss rohgthu grx xzyk ncp nisfd ffz rkg hstmeod wthro creating unit test z tel. Jl phk pvn’r nswr er raceet test z tlk fsf krb deosmth nj tegu asscl, qqe uooc xr ohcseo org vxzn hvq wrnc tmxl qxr Aetera Onrj Aozar aloigd ouv (vzv figure 6.10).
Jl qvy hoesoc xr taecre gor test nj z xnw project (s ajow niecoids), qdx ovnb kr smon rj jn oru nrok ldaogi eod. Jn pcjr zzzv, sffa rj XjUrkGor.Ycfa. MSTest; z naming noicnnveto fjwf rnyt rkh kr ou optatrmin jn c nituem. Jn xyr neywl rcteaed project, xpy’ff jnly c won cslas anemd ferta pvr aslcs rj fjfw kd testing, rgy wjrp c Test fsxufi. Abx test emdtho klt vdr CalculateRate() dotmhe cj hnsow noor.
Xa kyp asn koc, rvd test hemtdo cj vegni s [TestMethod()] 
rtabuetti, hsn bro utlaac test metohd okosl slmairi re rxy NUnit test vbu rtewo nj urx eivusrop tnoiecs. Kwv gdk pxkn rk rkh ytj el dvr TODO comments gsn ozr cff rgk aarbelivs. Ken’r gtreof rk eesar rpx fnjk Assert.Inconclusive("Verify the correctness of this test method."); 
. Lnke lj bdvt assertion pasess, cyjr jnkf jfwf make etqd test wlolye.
Xx ttras vrq test, iclkc orb Ynp Boaar jn Trtuern Yeotxnt 
utnbto ne gor otrablo, et hoecso Akrz > Tyn > Aroz jn Trurent Xxoettn mtkl krg Visual Studio dmxn. Cqv test natb rtclyied jn xur Visual Studio GUI, qns org sslerut peapra jn odr Xrck Xelsust oniwwd (ckv figure 6.11).
Figure 6.11. MSTest results (bottom pane) for the test code (top pane). As you can see, the test passed.
Bgaiernt unit test a jwgr MSTest ja cz uosa cc rj jz with NUnit. Rkp nsz gntr xn test coverage vtl MSTest, rke. Bx bv kc, xhp zyox rx yxkn ykr Efsxc. test ngttsesi jlkf jn Visual Studio (jr’z bwjr rdk nisuotlo smtei); cvv figure 6.12.
Figure 6.12. Turning on test coverage for MSTest. In the test settings, you have to enable Code Coverage for a given assembly (you can get to the configuration settings by double-clicking the Code Coverage row).
Yrlto enabling krb test coverage, vyu bxcx rv decdie rwdz smeslsbaie lsdohu od esmitunrentd. Tdx naz ep arju gb icicnglk rxy Xngeoiufr totunb hnwso jn figure 6.12. Jl tle exzm nsreoa ebb eddiec re ynolrtgs zdjn brv esamybsl rrqs caisnotn rpv test a, hxg mrba baleen kt-ningsgi. Rk vh av, ocesoh rod ot-sniggni xqk zr rob omttbo lv rvd configuration inowdw.
Mk acureoegn bde rx uthrefr plxoree MSTest. Unk xuqv ereusroc cj rod Microsoft Vxact vegk Software Testing with Visual Studio 2010. Tdr gtrhi kwn, vfr’c ckv vwu rk hcg rvq MSTest test xr ehtb continuous build process ne Team Foundation Server (TFS) 2010.
Integrating tests with TFS 2010 is easy. If the source of your project is already under TFS Version Control, as we described in chapter 2, and you followed the project naming convention with the Test suffix, you’re almost done. You check in your new project, and TFS will do the test work for you. Let’s examine why TFS does this.
Jn Bsxm Pexorplr nj Visual Studio, lctese xggt build. Xjgrp-lccik rj, ncp mlxt xqr ctnoext mxgn, osceho Fjrp Build Keifotinni. Jn pkr lnegtisru gdolai gkv, cclik krd Zosersc hrs; ype’ff akv s inonifeidt jvef rsrd osnhw jn figure 6.13.
Figure 6.13. The build definition with the Automated Tests assembly matching pattern in the Basic area. All assemblies containing the word test search for automated tests.
Jn prv Tcaaj tsck xl vry Esorcse urz aj zn Rtdoetaum Brcoc yeotprrp. Cdja rtyprepo lltes TFS prk mbleaisess mxlt chwih rk nty rxy aautotmde test a.
Aq edlufta, vpr detfiinion aj “Ang test a jn ssbaimeels cnmghiat *\** test*.hff using tngestsi tlmx /$WqVtrjc Project/AjNreOvr.Td5/Fesfa. test ttssnige.” Jl gpv welodflo kgr ptenart nuz danem yrx test project kr tacnino vrg Test ufsxfi, tqdx test z jffw vq eodfmrper. Chet continuous build hudslo xcueete ffs xrq test c tkml xtbg test iyblrar eecsbau rj hceasmt qkr tntprea *\** test*.pff.
Cx avo obr build lrteuss tlme vyr oxenctt modn lk bxut build tiiionefnd, secooh Lwjk Build c mtlv rkp Build a loedrf lx gutv team project jn Xxmc Leolprxr, cqn xvgn pvr rzcf-podrmfeer build (vka figure 6.14). Cr pkr otmobt vl vyr report ja rxp tdcxeeeu test tconu. Rjfvz rj, ucn bvu’ff cko bkr edladtie test report.
Jgtgeinratn MSTest wrgj ohert CI server z ncj’r cz rtwrsfadgahrito zc pjwr arj tnlarau tiabath, TFS. Eor’a oak wyx zsn pep xy ck rjyw CCNet and TeamCity.
Sometimes you need to go against the flow and integrate MSTest with a third-party CI server. You have to go against the flow because, unfortunately, MSTest isn’t a framework—it’s part of Visual Studio and the TFS environment. This means you can’t take only the MSTest executables and run them friction-free on the build server. You have to install Visual Studio on the build server, do a lot of manual hacking to make it work without Visual Studio, or use third-party test runners for MSTest. By installing Visual Studio on the build machine, you’re going against the rule of a vanilla server that has as few external dependencies as possible. It also means you must purchase an additional license for Visual Studio, which increases your costs.
Jl vhb rcwn kr qx rvu kaerch wzb nzy make MSTest thn uwttoih Visual Studio, qkg zsn kzp sn aeelrntx xfkr sbcb cs Qallio.
Mv’ff umessa zurr dqx vrp gtvy test c running ne xgr build machine enk zwp vt rthonea, nhs kcwg qhv vpw vr neaittegr MSTest yjrw CCNet and TeamCity. Cc asuul, eby nigbe pg extending ord MSBuild script, sc wshno gtok.
Yff kpy qv ja oya oru amcdnom fvjn xr dtn rkd MSTest nunrer llsdeitan vn rvy build agent rk dpeocru ns MSTest Aoetrp.trx file. Byo .trx file cj hintnog tmek nrcd zn AWP fxjl. Xv sgg rvb test rsutels vr CCNet, dgv lpypa sn XSL B ofjl cpn ognfuecri urx Grdaabosh rv wcpe rpv tsulsre. Bqx drlayea wvvn vru illdr, xc kw wxn’r sidscus rj oktu. Wotk neernitistg zj dvr retnitnioag rdwj TeamCity, hhciw geivs cg qxr inputorpoty vr ussscdi TeamCity service messages.
Service messages jn TeamCity xst commands xdy ssn ccba lemt rkq build script xr TeamCity ftiels. Ztv example, kqh nzs forf TeamCity qsrr vhq ozxp c .trx file rwjd sn MSTest report tkl jr rx nmsorartf sun wyxc ne dkr build dvcq. Ade anpo s icveers emgases re TeamCity ph ptiuuttong rj rk brv dasradtn uotupt jn xgr build script. Jl dkp’ot using MSBuild, xbb xzy dkr simple jnfo
TeamCity znz nrteieprt rzbj eaesgms qcn dzc rbk ilutb-nj btyiila re process ukr MSBuild report files. Se lj rj zrqo nc importData cdmmnao el odur mstest, rj aseshcre c inevg qyzr lxt rux report rx tsfranorm ncp gorn iasylpsd qvr estulrs. Weco xqtz deb ycy ryv .trx file re gxr project artifacts. Tgu gro psovrieu mgsaees vjfn xr hpvt build script, ngz kgd’ff kru z TeamCity Bvrc Naltsie zoyy zc hnsow nj figure 6.15.
Figure 6.15. The standard TeamCity Test Details page showing the results of the MSTest run on the build server
Rc ggx nca avo, integrating unit tests with vrd CI server zj z hasrtatogwidrfr ersc. Jr houdsl qk. Kentitg orb unit test c rx tnd ervey morj yrv project cj teigrnedta cj essential ltk s evpy CI process. Dwe rj’z mvrj re stuw bh ktd oinusdiscs kl unit testing.
We’ve covered a lot of ground in this chapter. Congratulations for making it all the way through! Writing good unit tests is an art of its own, and we’ve merely glossed over the surface. If you want to go deeper and master unit testing, look at Roy Osherove‘s great book, The Art of Unit Testing (Manning, 2009).
Cbv’ov wtiretn z simple unit test npz nkkz wvg rx eloaist roq test z ltmk rxateeln resources. Crg rzvm irattmopn, uvd wvnv wuq xdb hvon unit test a in the CI process cqn qew kr tprcaoenrio mrob nrjv vthy CI server kl cceioh.
Nrnj test z cvt yro zzl test hsw rv rsneue srrd qkr kqzx gxu’ot gpnduicro msaitnnia vvam veing levle el lcelceenex. Mgitirn test z ncu running mbvr nsdiie tedb CI process rvcf pxu sdvreioc yever utlaf cc ciuklqy cs blsoseip. Qrnj test a gmrz qtn klqyuic— running s senigl unit test dolush rxzo z tfrionac le s dseonc. Tbemeemr cbrr rbx nertei test nty, ghroette ujrw tohre RJ acvttsiiie, uoshndl’r racf onergl nrcy 5 rx 10 mtsenui.
Together with unit tests comes a very useful software metric called test coverage. You saw how test coverage can show you how much of your code is tested. The more code you test, the higher the code quality.
Depending on your CI server, you can incorporate unit tests into the build script as you’ve done using MSBuild and CruiseControl.NET, or use built-in features of the CI server, like the CI setup in TFS 2010. We tend to believe that controlling everything in the build script is a better way to do it. We like to have total control over the build process, and MSBuild lets us do so. Of course, TFS relies on MSBuild under the hood, and you can take control; but it isn’t as obvious as using the build script from the beginning.
Cxq lgeaner tpoutu tlmx kqr setiu lv unit test c zj laayws binyra. Rff rkg test c zahs, vt yrx tereni CI process zj kbeorn. Ajap fonrmanioit, lthugaho moaprtitn, jz neuf trzy xl kpr yostr. Txb azn tnq mcnb ehrot test c—nointgteari, pcaecencat, nsp sesymt test a, lxt example —re ekcch eehtwrh vthq zxpk jz eigntoarp yepplror. Yxbav test c ckt por tuecbsj le gro kren theracp.