微软软件实现技术 (4)

Testing IntroductionIvan Hoivanho@Test LeadSelf Introduction•8+ years in Microsoft•Started as a contractor •Products/Projects I have worked on –3 version of BizTalk Server–PSS crawling system–NLG APIs–Office Communicator Web Access 2007–Office Live Meeting 2007Agenda•Career Of a Tester •Testing Your Testing Sense •What is in a Test plan?Career of a TesterCareer PathPossible Path 1:STE -> SDET -> Senior SDET -> Technical Lead -> Test ArchitectPossible Path 2:STE-> SDET -> Senior SDET -> Test Lead -> Test Manager -> Test DirectorExample:•Grant George–Corporate Vice President, Office -Testing and Operations –/presspass/exec/george/default.mspxTesting Your Test Sense! #1•Please test the function:–int StrToInt(char str[]);•Example StrToInt(“1234”) should return 1234Testing Your Test Sense! #1•Test cases–Check for value within Integer range–Counting separator (for example 1,000 or 2,111,234)–Scientific number (1.9950631168807583848837421626836e+3010)–Check for null–Check decimal number–Check with Spaces–Check with Empty String–Check overflow ( Stress )–Check non numerics–Check performance–check with zero ( for example 012345 )–Check with arithmetic symbols ( +/-, etc.. )–Globalization ( Europe uses ‘.’instead of ‘,’as separator )What is the problem?•Randomization!–Which implies you may miss a lot of cases •Solution: need to be SystematicCategorization •Valid vs. Invalid •Normal vs. unusual •Performance related •Boundary related •Internationalization relatedCategorization•Valid–Negative number–Positive number–0–With leading or trailing 0’s–With spaces in front or trailing spaces –Empty String•Invalid–Non-numerics•Abcde•@#&($@#&(–Decimal points•1.00345Categorization•Normal–1234–1234.5–-123–0–1,000–.444–+123–-367.0•Unusual–0 0 0 234–一二三– 1.9950631168807583848837421626836e+301–1+1Categorization•Boundary–Boundaries of Int Type ( differentProgramming languages maybe different )–Human boundary•0•Negative number•Increase in digitsCategorization •Performance•Using progressive increase numbers to plot agraph [ O(n), O(n2), O(log n)…]Categorization •Internationalization–Separator–Decimal Points–Different languages OS–Language specific bits testingTesting Your Test Sense!•Test cases–Check for value within Integer range [Normal, Valid]–Counting separator (for example 1,000 or 2,111,234) [Valid?, Nor mal?]–Scientific number (1.9950631168807583848837421626836e+310) [boundary,unusal?]–Check for null [Valid, Boundary]–Check decimal number [Invalid,normal]–Check with Spaces [Valid?,normal?]–Check with Empty String [Valid, Boundary]–Check overflow ( Stress ) [Boundary]–Check non numerics [Invalid]–Check performance [Performance]–check with zero ( for example 012345 ) [Valid,normal]–Check with arithmetic symbols ( +/-, etc.. ) [Valid,normal]–Globalization ( Europe uses ‘.’instead of ‘,’as separator[Valid?,unusal?,Internationalization]What is the problem?•Blackbox testing!–Which implies you are guessing a lot–Solution: combine with white box testingTesting Your Test Sense! #2•Int StrToInt(char str[]){int i = 0, num = 0;while (str[i]) {num * = 10;num += (str[i++] –‘0’);}return 0;}Testing Your Test Sense! #21.Int should be int2.Didn’t handle negative number3.Didn’t return a value4.Didn’t check for null value5.Didn’t check for whether it is within integer rangeTesting Your Test Sense! #2•int StrToInt(char str[]){int i = 0, isNeg = 0, num = 0;if (str == null) return 0;if (str[0] = ‘-’) {isNeg = 1;i = 1;}while (str[i]) {num * = 10;num += (str[i++] –‘0’);}if (isNeg)num *= -1;return num;}Conclusion•Blackbox and Whitebox testing should co-exist•Consideration:–Time–Effectiveness–Technical backgroundTesting Your Test Sense! #3Testing Your Test Sense! #3What are the suspects:-Machines itself?-Compatibility with OS?-Compatibility with other software?-Network-Network bandwidth?-Firewall?-Server?-Bad input to the Server?-Capacity of the Server?-Receive mechanism on the Server?-Send mechanism on the Server?-Bad output by the Server-Client?-Does request go out from Client A?-If not is it problem with UI?-Or is it problem with the sending mechanism?-Does response send to Client B?-If so is it UI problem?Summary•Know how to diagnostics •Nail down the root causeTest Plan : Testing MSNMessengerTesting MSN MessengerMSN MessengerTesting MSN Messenger •Breaking down a complicate problem into small/isolate problems–Unit Testing•(most Basic verification on Classes, Functions,Methods )–Componentized Testing•Feature/functional testing–( Authentication, Contact Management, Search, IM )•Client and Server separation•UI and business logic separation–Only if developers separate the UI and logic well enoughTesting MSN Messenger •Think outside of its features and functionalities, testing its limitation –Performance–Stress–Longevity–Fuzz–Error handlingTesting MSN Messenger•A software can be used by everyone and think like one–E2E testing–Usability–Globalization–LocalizationTesting MSN Messenger•List out all dependencies–Integration testing ( Schema testing,Federation testing )–Topologies testing•Single Forest, Multiple Forest, Behind a Firewall,Proxy Server–Platform Compatibility testing–Backward compatibility testingTesting MSN Messenger •A Trustworthy software–Security testing•Privacy•Denial of Service•Client end attack•Stolen you IDsTesting MSN Messenger •Beside executing test cases –Code review–Review design and architectureTesting MSN Messenger•How can you be confident about the quality?–Code coverage–Meeting Defined Exit Criteria–Dogfood–Bug Trend–Bug per line of code–Customer Feedback ( From Alpha or Beta orprevious version )Before Writing a Test Plan •Understand Customer Requirement –Functional Specification •Understand the Architecture and Design ofthe Overall Product•Understand the Technologies being used •Research and Investigation on test frameworks and toolsWithin a Test Plan •What do we need to focus on?–Why?–How?•Break Down your Focus AreasAfter Writing a Test Plan •Review with others•Prioritization•Request for Resources ( Human, hardware and so on )Exercise…•Please come up with a Test Plan for Microsoft CalculatorThank You。